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A METHOD FOR TRACKING DEPTHS IN A SCANLINE 
BASED RASTER IMAGE PROCESSOR 

CROSS-REFERENCE TO RELATED PATEPW APPLICATIONS 

This application claims the right of priority under 35 U.S.C. § 119 based on 
AustraHan Patent AppUcation No. 2003903448 ffled June 26, 2003, which is 
incoiporated by reference herein in its entirety as if fully set forth herein. 

COPYRIGHT NOTICE 
This patent specification contains material that is subject to copyright protection. 
The copyright owner has no objection to the reproduction of this patent specification or 
related materials fi-om associated patent office files for the purposes of review, but 
otherwise reserves all copyright whatsoever. 

TECHNICAL FIELD OF THE INVENTION 
The present invention relates generally to rendering graphical objects and, in 
particular, to the resolving of z-orders of graphical objects to accelerate rendering. 

BACKGROUND 

Raster image processors are fimdamental to the field of computer graphics. A raster 
image processor (RIP) takes geometiic primitives as input, and produces an array of pixel 
values on a raster grid as output. In this document, 2D rendering systems will be 
considered. 

Two important classes of RIPs are object based RIPs and scan line based RIPs. The 
difference betwerai these, two classes of RIPs Ues in their inner loops - that is, which 
entities their algorithms iterate over. Object based RIPs iterate over all fixe primitives 
they have to render, whereas scan line based RIPs iterate over each scan line to be 
rendered, in raster order. 

Consider the case of rendering a set of polygons, differentiated firom each other by 
their outlines, fills and rendering deptiis (ie. z-order). An object based RIP would 
typically iterate over the set of polygons in z-order, rendering each polygon in timi. This 
is a "Painter's algorithm" ^proach. In conti-ast. a scan line RIP considers each scan line 
in turn, determining where the edges of each polygon are on the current scan line. The 
spans of pixels between the edges which intersect tiie current scan line are then filled. 

A refinement to the scan line based rendering system lies in the use of an active 
edge list. In such an approach, instead of considering the edges of every polygon on 
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every scan line, the scan line RIP maintains a list of only those edges that intersect the 
cunent scan line, and tracks those edges fiom scan line to scan line. This ^roach 
typically makes use of forward difference techniques, in which the edges are tracked by a 
form of Bresenham's scan-conversion algorithm. 

At the start of rendering each scan hne, new edges may be added to the list of active 
edges. Sorting is important here, as it dramatically cuts down the number of edges that 
need to be considered for addition to the active edge list. Specifically, the total set of 
edges a usually sorted in raster order of starting points. By starting point, we mean the 
termmal point first encountered in an entire raster scan. This allows the rapid 
determination of which edges, if any, are becoming active at the start of a scan line. At 
the end of a scan hne, if an edge is determined as not mtersecting the following scan hne, 
that edge is removed firom the active edge Ust. 

Consider a typical scan hne under the above RIP. Given that the x-ordinates of aU 
the edges that intersect this scanline are known, and that it is desired to output raster 
image data for this scan hne. the next problem to be solved is to determine the raster data 
content of the spans of pixels between the crossmg points. That is. the span between 
contiguous crossmg pomts in x-order wiU map &om raster hnage space to a single set of 
polygons in a graphic model space. This approach is with the allowance that the RIP does 
not handle antiahasing. Imphcit in this approach is that overlapping polygons are 
distmguished from each other by their depth. That is. in the set of polygons exposed in a 
span between crossing points, the uppermost polygon, if it is fiilly opaque, will be the 
only polygon to be rendered over that span. If the uppermost polygon less than fvilly 
opaque (ie. contains some transparency), lower polygons will contribute to the span. 

Scan hne rendering avoids the need for a screen (frame) buffer to accumulate the 
results of rendering, this bemg a feature and deficiency typical of object based (Painter's 
algorithm) rendering systems. However, a scan Une rendering system can be fiirther 
characterised at this pomt by considering how it solves the above problem. Specifically, 
some scan Une rendering system have reduced the need for a fijll screen buffer down to a 
scan line buffer, while others have avoided the need for a scan hne buffer, and can render 
directly to an underlying output raster data buffer. 

It is appropriate to consider some different line buffer RIP models. Firstly, there are 
those RIPs that maintam a Une buffer of raster image data. Edges are sorted in y-ordinate 
(ie. scan Une) but not m x-ordmate (pixel location within the scan hne), and spans are 
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accu.„ulaw in a random patten, into mebufier. Tl.a. is. the orter in which of 
.pans occurs is no. related to tteir x order. I, snoh a system, ttere is no concept of z- 
order m which higher greyscale output values take pr^edence over lower values when a 
P«el ,s written to. Tire Un= buffer is written out to the output buffer in x^rder after aU 
spans have been accumulated lor the current scanline. 

A variation on the above RIP model is feund in other scan line based MPs, which 
mamtain a line bufler of crossing points instead of raster data. Again, edges a« sorted m 
y brt not m x. For each scan Une. aU dre crossing points are detemuned. and then sortod 
by X. to odrer words, a Ime buffer of crossing pomts is maintained, not a hue buffer of 
raster data. Noto again however to the line buffer of cussing messages is accumulated 
in random order. 

A third class of scan hne based RlPs do not have either a line bufto „f raster data or 
a hne buffer of crossing messages. Such are avoided by a full raster order «,rting of the 
aU edges prior to ™,dering. That is. an »dges are sortod by starting XK^to as well as 
y-ordu^to. As each pixel of a scan hne is considered, the Mst of edges is checked for 

edges that are becoming active. 

We will now consider the issue of detemuning the raster dato content of a span on a 
^ven scanline - a problem common to aU scan hne renders that support overlapping 
polygons separated by depth. As introduced above, a constant set of polygon fills ■ 
orfer«l by depth. potentiaUy conMbute to .he span, "nre problem can be considered m 
.«ms of fills and the d^ths at which they occur, assummg each polygon has a smgle fiU 
Which inherits the polygons depth. 

to systems that consider spans in x-order. solving this problem implies the 
mamt^ance of a table of currently active depths as rendering progresses fiom span to 
span. Ingress flom span to span occurs on any pixel that is mossed by an edge Ttatis 
a span is an interval on thecurrent scanHne over which the set of currently active depths' 
.s constant. By active, it is meant that a conceptoal shoe through aU the polygons which 
mt««ct the span would mclude all the active depths (ie. all graphic objects that 
contnbute or influence the final pixel value). 

Tie set of active depths aUows the detennmation of the raster output for a span if 
a.e set of active depths is maintamed in decreasmg depth order. If fiUs are no. aUowed to 
mclude transparency, the fiU at the head of the toble of acUve depd^s win be uppennost 
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and be *e only flu conttbuting to a.e ra^ fo, ae fill. If tanspa^cy is allowed 

the fiUsassocia.«lwia, depths IbUowtog in actable may abooortribute to raster o»4,«" 
A solufon to the problem of maintaining this table of cmrenUy active depths exist 
m pnor art. For example, a complete status table of all the dq,ths that exist for the page 
»age to be rendemi can be constructed. This approach is characterised by ±. fee, 
ate depths exist in a linearly addressed memory in which the depths are impUci. in a,e 
location in memory where fiB data may he stored, if a fill exists at that depfl.. m other 
words, there is a complete set of slots for pointers to fiU infonnahon. one for each depth, 
and the slots are arranged in depth orfer. Insertion of a fill into such a table is trivial as 
there will always be a slot for a given depth. However, ae pattem of slot ocoupafion win 
typicaUy be spare. In practice, when considering the fiU and depth referenced by an edge 
crossing, the depth of the fiU is used te index this complete table of depth slots. This is an 
essenhaUy an order one operation, . aUowing the RIP to opiate at real-time rates 

A cache, which may be tem.ed a summary table, can be used to speed up a lookup 
to such a complete table, Thia is typically necessary because the total number of depflas 
that «ust for a page may he very large. For example, some real-world PostScript print 
rendering jobs have extremely large numben, of depfl^s. Anoflaer consequence of the very 
large size of the complete depth slot table is that hardware implementations of this RIP 
model cam.ot often maintain the complete Ust inten^ally. The table must therefore be held 
m external memory, which is slow to reference. If more depths exist than can be 
accommodated by the table of depth slots, additional tables can be swapped in, at .he cost 
of Significant additional complexity. 

Further, a RIP with this model is not readily suited to animation, being ^Hertions 
where the set of depths occupied by fills changes ftom ftame to flame. Also, if the input 
to the animating tenderer is not drtemunistic the set of occupied depths may also not be 
detenmmstic. Many depfl. slots will have to be put aside for fills fom«i on fut»e frames 
Alteniatively, fills may be given difier«tt depfl,s ftom frame to ftame, but m the class of 
pnor art RIPs cmrently being examined, a fill is a tightly conpled m combination to its 
corr^pondmg depth. Hat is, a fill has an impUed depth by being a. a certain memory 
locafon. I, follows that reassigning fills can entail substanfial data movement, which is 
undesirable, 

A fl«her deficiency is found to mamtainfag flie table of acHve depths m the context 
of compositing. Inserting a new flu toto the summary table is not expensive, as it is as 
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deep as the table of aU depth slots found on the page, and each fill has a coiresponding 
waiting slot. However, when a span is to be composited, the operation is costly, since the 
summary table is both very large and usually very sparse, while the set of fills that the 
compositor needs to consider is generally comparatively small, and is compact. In 
practice much complexity and siUcon is expended in a depth encoder that is as deep as the 
status table. This encoder is then used to create a compact set of fiUs to be composited 
over the span in question, by the process of considering the set of active fills ordered by 
depth detailed above. 

SUMMARY OF THE INVENTION 
It is an object of the present invention to substantially overcome, or at least 
ameliorate, one or more disadvantages of existing arrangements. 

According to an aspect of the invention, there is provided a method of rendering a 
scan Une of a graphic object image in a scan Une Tenderer for a span of pixels lying 
between two x-order consecutive edges intersecting said scan line, said method being 
characterised, for said span of pixels, by maintaining a subset of depths present in the 
rendering, said subset being those depths that are present on said span and being 
maintained in depth order and subject to removal of depths where the coiresponding 
depfh is no longer active. 

Preferably, the subset of depths is maintained using a content addressable memory. 

According to another aspect of the invention, there is provided an apparatus for 
implanenting any one of the aforementioned methods. 

According to another aspect of the invention there is provided a computer 
program product including a computer readable medium having recorded thereon a 
computer program for implementing any one of the methods described above. 

Other aspects of the invention are also disclosed. 

BRIEF DESCRIPTION OF THE DRAWINGS 

At least one embodiment of the present invention will now be described with 
refer^ce to the drawings, in which: 

Figs. 1(a) to 1(c) illustrate active edge determination at the sub-pixel level; 

Figs. 2(a) to 2(d) show an example of edge processing for a scan line at the sub- 
pixel level; 

Fig. 3 is a flowchart of a method of step 252 of Fig. 23 for generating crossing 
messages; 
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Fig. 4 depicting a method of bitmap image pixel generation; 

Fig. 5 shows a C language source code of a modified Bresenham's algorithm; 

Fig. 6 is a flowchart of a bottom-up compositing process; 

Fig. 7 shows an example Bezier curve related to the tracking parameter generation 
process of Fig. 41; 

Fig. 8 depicts the coping of winding counts for sub-pixel values in the example of 

Fig. 2; 

Figs. 9(a), 9(b) and Fig. 10 illustrate different compositiug scenarios; 
Fig. 11 is a flowchart of step 113 of Fig. 13; 

Figs. 12(a) to 12(d) provide a graphical indication of coverage and A-buffer 
operations; 

Fig. 13 is a high-level flowchart of a compositing ^proach used herein; 
Fig. 14 is a circle that iUustrates operation of Bresenham's algorithm; 
Fig. 15 is a flowchart of step 465 of Fig. 46 for converting an ordered set of 
coordinates into an ordered set of edges; 

Figs. 16(a) to 16 (c) iUustrate how 2-dimensional primitives combine to form a 
graphical object; 

Figs. 17(a) and 17(b) illustrate the relationship between sprites and transformation 
matrices; 

Figs. 18(a) and 18(b) illustrate end-caps of a stroke; 
Fig. 19 shows a generalized ellipse; 

Fig. 20 illustrates determining the circle that underlies an elUpse; 
Fig. 21 is an example of re-ordering active edges at crossings; 
Figs. 22(a) and 22(b) depict data flow for the edge processing module of Fig. 56; 
Fig. 23 is a flowchart of step 257 of Fig. 24 showing edge processing for a single 
scan line; 

Fig. 24 is a flowchart showing edge processing for a single fiame; 
Fig. 25 illustrates operation of a gradient fill look-up; 
Fig. 26 illustrates generating crossing messages for glyphs; 
Fig. 27 illustrates various gradient tracking parameters; , 
Figs. 28(a) and 28(b) illustrate various fills for types of end caps; 
Figs. 29(a) to 29(c) illustrate respectively two prior art error diftusion approaches 
and an error diffusion approach used by the pixel extraction module of Fig. 56; 
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Fig. 30 is a flowchart of the error diffiision process (half toning) of the 
module 618; 

Figs. 31(a) to 31 (c) iUustrate the z-level relationship between sprites and local 
graphic objects; 

Fig. 32 is a flowchart for determining an index to a gradient look-iq) table; 
Figs. 33(a) to 33(c) iUustrate left and right fills used to create the object of Fig. 16; 
Fig. 34 depicts various image spaces used in the present rendering system; 
Figs. 35(a) to 35(c) depict sets of coordinates in a moiphing process; 
Figs. 36(a) to 36(c) show an example of a stroked path; 

Fig. 37 is a flowchart depicting operation of the Pixel Extraction Module to output 
pixels to a frame buffer memory; 

Figs. 38(a) and 38(b) are flowcharts depicting operation of the Pixel Extraction 
Module to output directly to a display; 

Fig. 39 is a processing flow representation of the Pixel Generation Module; 

Fig. 40 is a flowchart of the processing of a run of pixels to generate output color; 

Fig. 41 is a flowchart for the generation of tracking parameters for quadratic 
Bezier curves; 

Fig. 42 is a flowchart of an incremental approach radial gradient determination; 

Figs. 43(a) and 43(b) illustrate different fiU results arising from the non-zero 
winding, negative winding and odd-even fill rules; 

Fig. 44 illustrates calculating absolute depths in a display list; 

Figs. 45(a) and 45(b) depict edge management for stroking a join; 

Fig. 46 is a flow diagram showing the processing performed on each ordered set 
of coordinates by the Morphing, Transform and Stroking module; 

Figs. 47(a) to 47(f) depict operation of the radix sort algorithm; 

Fig. 48 illustrates the contribution of opacity at sub-pixel levels; 

Figs. 49(a) to 49(f) show examples of stroking edges; 

Figs. 50(a) to 50(e) show examples of stroking and transforming edges; 

Figs. 51(a) to 51(e) iUustrate left and right fiUs for stroking edges; 

Figs. 52(a) and 52(b) Ulustrate operation of the Z-level activation table; 

Figs. 53(a) and 53(b) show an example of reconfiguring the Z-level activation 

table; 
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Figs. 54(a) and 54(b) are examples of updating the Z-level activation table for Z- 
levels of changing interest; 

Fig. 55 is a flowchart of a top-down compositing process; 

Fig. 56 is a schematic block diagram representation of thin cUent imaging engine 
according to the present disclosure; 

Figs. 57(a) and 57(b) show conversion of S-buffers to A-buflFers according to the 
three winding rules; 

Fig. 58 is a flowchart illustrating operation of the Z-Level Activation module; 
Fig. 59 shows a Ust of interesting Z-levels maintained by the Z-level Activation 
Module; 

Fig. 60 is a schematic block is a schematic block diagram of a computer 
arrangement upon which some arrangements described can be practiced; 

Figs. 61(a), 61(b) and Fig. 62 depict compositing approaches using top-down and 
bottom-up sectionaUzation of a compositing stack; 

Fig. 63 shows different stoke being a^jplied to discrete portions of a path; 

Fig. 64 shows an alternate form of top-down compositing; 

Figs. 65 A and 65B depict an update of the z-level activation table across a scan 

line; 

Fig. 66 illustrates crossing-message generation for a glyph edge; 
Fig. 67 illustrates a process for spUtting an ellipse segment into segments 
monotonic in y-ordinate; 

Figs. 68 A to 68C illustrate an animation sequence of image frames; 

Figs. 69A to 69C illustrates the processing of new and static edge buffers for the 
production of the animation sequence of Figs. 68A to 68C; 

Fig. 70 illustrates the updating of S-buffers for zero-crossing messages; and 

Fig. 71 is a flowchart sunilar to Fig. 58 but showing operation of the Z-level 
activation module to discard levels from the list. 

DETAILED DESCRIPTION INCLUDING BEST MODE 
Table of Contents 

1. Introduction 

1.1. Coordinate spaces 

1.2. Graphic objects 

1.3. Glyphs 
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1.4. Z-levels 

1.5. Stroking 

1.6. Moiphing 

2. The Driver Module 
5 2.1. Sprites 

2. 1 . 1 . Sprites: transformation matrices 

2.1.2. Graphic objects and their depth 
2.2. The display Ust 

2.2.1. Frame rendering 

10 2.2.2. Graphic objects and z-levels 

2.2.3. Local depths and absolute depths 

3. Transformation, Morphing and Stroking 

3.1. Morphing 

3.2. Transformation 
15 3.3. Generating Edges 

3.4. Decomposition of strokes into edges and z-levels 

3.4.1. Stroking a straight edge 

3.4.2. Stroking a curved edge 

3.4.3. Stroking a join 

20 3.4.4. Stroking Equal or Opposite Edge Joins 

3.4.5. Generating end-caps at the end of a path 

3.4.6. Endcaps between stroked and unstroked edges 

3.4.7. Z-level assignments for opaque strokes 

3.4.8. Z-level assignments for transparent strokes 
25 3.4.9. Transformation of Stroking Primitives 

3.5. Filtering 

3.6. Generatmg edge-tracking parameters 

3.6.1. Generating straigiht-edge tracking parameters 

3.6.2. Generating quadratic Bezier curve tracking parameters 
30 3.6.3. Determining the sign 

3.6.4. Generating Elliptic arc tracking parameters 

3.6.5. Generating Glyph edge tracking parameters 
4. Sorting 
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5. Edge processing 

5.1. Input and output 

5.2. Top-level Operation 

5.3. Active Edge Tracking 

5.4. Example of Edge Processing 

5.5. Converting edges into active edges and edge persistence 
5.5.1. Static Edge PersistOTce 

5.6. Active Edge Processing 

5.7. Tracking Edges 

5.7.1. Tracking Straight Lines 

5.7.2. Tracking Quadratic Beziers 

. 5.7.3. Tracking Quadratic Polynomial Fragments 

5.7.4. Tracking Elliptic Arcs 

5.7.5. Tracking Glyphs 

5.8. Anti-aliasing and Crossing Message Generation 
5.8.1. Generating Crossing Messages for Glyphs 

5.9. Example of Crossing Messages 
5.9.1 Another Example 

5.10. Re-ordering of Active Edges and Crossing Messages 

6. The Z-level Activation Module 

6.1. The Ordered Set of Interesting Z-levels 

6.2. The Flow of Control in the Z-level Activation Module 

6.3. Activating and Deactivating Z-levels: Winding Counts 

6.4. The Ordered Set of Interesting Z-Levels - continued 

6.5. Adding new Z-levels to the Ordered Set of Interesting Z-levels 
6.5. 1 . Maintaining an Ordered Set of Interesting Z-levels in Hardware 

6.6. Processing Runs 

6.7. Converting S-bu£Fers to A-buffers: Winding Rules 

6.8. Processing Runs, Continued 

7. Compositing Module 

7.1. Intermediates 

7.2. Z-level Fills 

7.3. Basic Flow 
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7.4. Graphical Overview 

7.5. Contribution Calculation 

7.6. Bottom Up Composite 

7.7. Top Down Composite 

5 7.8. Alternative Compositing Approaches 
7.9. Top-Down Advantages 

8. Pixel generation 

8.1. Linear Gradient Pixel Generation 

8.2. Radial Gradient Pixel Generation 
10 8.3. Bitmap Image Pixel Generation 

9. Pixel Extraction 

9.1. Liputdata 

9.2. Output to frame buffer 

9.3. Output direct to display 
15 9.4. Halftoning 

10. Implementation 

1. INTRODUCTION 

This document describes a Thin Client Imaging Engme (TCIE) which is a system 
for rendering 2D gr^hic objects, using minimal computing resources. Examples of 

20 where such resource levels may apply include portable devices or those with smaU 
displays, such hand-held computing devices including mobile telephone handsets and 
games, and office equipment such as printers and copiers. A top-level diagram of the 
TCIE system 699 is shown in Fig. 56 in which the TCIE system 699 is structured as a 
pipeUne of processing modules. Each module will be described in order of the flow of 

25 data through the system. The modules are conceptually divided between a Display List 
Compiler 608 and a Rendering Engine 610. The Display List Compiler 608 prepares 
information describing the desired output, and the Rendering Engme 610 uses this 
information to generate that output image (e.g., rendering to a display device or frame 
buffer). The TCIE system 699 can be xised to generate a series of temporally spaced 

30 output images, such output images referred to hereafter as 'frames'. This use of the TCIE 
system 699 creates the effect of an animation (or 'movie') being played on the output 
display. 
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1.1. Coordinate spaces 

Referring to Rg. 56, the first module of the system 699 is the Driver Module 615 
which maintains collections of graphic objects and information about them. Fig. 34 
describes the spaces used by the system 699. Fig. 34 initiaUy shows a graphic object 
described in object space 335. Next, the same graphic object is shown transformed into 
global logical space 336. Next, the same graphic object is shown transformed into a 
render space 337. and finally the same graphic object is shown transformed into the 
display space 338. 

The transformation fi-om object space 335 to global logical space 336 is achieved 
by the graphic object's placement transform. This placement transform may be a product 
of a hierarchy of transformation matrices, to be described later. The transformation from 
global logical space 336 to render space 337 is achieved by a viewing transform which 
converts global logical coordinates to sub-pixels (for the purpose of anti-aliasing). The 
transformation from render space 337 to display space 338 is achieved by an anti-aliasing 
process that produces display pixels from constituent sub-pixels. In the degenerate case 
of one-by-one anti-aUasing, the render space 337 and display space 338 are the same, i.e., 
the logical space 336 can be transformed directly to the display space 338. 
1.2. Graphic objects 

The input into the system 699 consists of a set of graphic objects and associated 
metadata. Fig. 16(c) shows a graphic object 171 rendered onto a display, with its 
corresponding components being shown in Fig. 16(a) and Fig. 16(b). Graphic objects are 
two-dimensional display primitives described by an ordered set of one or more of the 
following drawing primitives: new drawing positions, straight lines and curves. The 
drawing primitives describe parts of the outhne of a graphic object. Each primitive is 
associated with one or more coordinates. New drawing positions may be specified as 
either an absolute ofi&et fix)m the object's origin or relative to the endpoint of the previous 
primitive. New drawing positions are described by a single coordinate, straight lines are 
described by a pafr of coordinates, and curves are described by a sequence of three 
coordinates. Straight lines use the pair of coordinates to define the start and end points of 
the line. Curves are implemented as quadratic Bezier curves, wherein a first coordinate 
defines the start of the Bezier curve, a second coordinate defines a control point, and a 
third coordinate defines the end point of the Bezier curve. Bezier curves are well known 
to those skilled in the art. 
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The coordinates of edges are stored as an ordered set of relative coordinates, 
which reduces memory storage requirements and also determines the direction of edges. 
It is impUed that the first co-ordinate of a straight line or curve is the last co-ordinate of 
the previous drawing primitive. TABLE 1 is an example of an ordered set of primitives 
that could form the display object shown in Kg. 16(c). Ih this example, Y ordinates 
increase downwards, and X ordinates increase to the right Also, the terms for new 
drawing position, straight lines and curves are MOVETO_ABS, MOVETO_REL, 
LINETO and CURVETO, respectively. The start point in this example is (0, 0) 14lT 

TABLE 1 



Primitive Type 


Coordinates (relative, unless MOVETO_ABS) 


MOVETO_ABS 173 


(0,0) 141 


MOVETO_REL 174 


(40,-50) 177 


LINETO 157 


(0, 80) 142 


LINETO 158 


(10, 0) 143 


LINETO 172 


(0, -50) 144 


LINETO 1 


^JU, V) 145 


LINETO 160 


(0, 50) 146 


LINETO 161 


(100, 0) 147 


LINETO 162 


(0, -80) 178 


LINETO 163 


(-30, -30) 148 


LINETO 164 


(-80, 0) 149 


LINETO 165 


(-30, 30) 177 


LINETO 166 


(140. 0) 178 


MOVETO_REL 175 


(-30,40)176 


CURVETO 168 


(0, -20) ISO then (-20, 0) 151 


CURVETO 167 


(-20, 0) 152 then (0, 20) 153 


CURVETO 169 


(0, 20) 154 then (20, 0) 155 


CURVETO 170 


(20, 0) 156 then (0, -20) 176 



1.3. Glyphs 

Glyphs are special types of graphics objects with the ftffther restriction that they 
are always drawn directly into display space. . Glyphs are designed for situations where 
the shape that is being rendered is: 
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(i) small, and 

(ii) designed to be placed on exact pixel boundaries. 

Examples of shapes that are weU-suited to be represented by glyphs include hinted 
font characters. 

Instead of a path, glyphs are represented by a one bit-per-pixel bitmap with two 
associated fills. TTus bitmap acts like a mask - where bits are set, the "on" fiU is 
displayed; and where bits are not set. the "ofi" fill is displayed. 
1.4. Z-levels 

A z-level is a display primitive used to describe how part of the display enclosed 
by a subset of a graphic object's edges should be colored. For example, a z-level could 
describe the enclosed area as being filled with a solid color. A z-level is also assigned an 
absolute depth, this absolute depth bemg an integer value used to specify which z-level 
should appear on top of which. A z-level with a higher absolute depth is rendered on top 
of a z-level with a lower absolute dq)th. 

Up to two z-levels are associated with straight line and curve drawing primitives - 
a possible first z-level to be rendered to the left of the drawing direction of that primitive 
and a possible second z-level to be rendered to the right of the drawing direction of that 
primitive. Figs. 33(a) to 33(c) demonstrate this concept by describing the left and right 
fills used to create the graphical object 331 seen previously in Fig. 16(c) as the 
object 171. 

Fig. 33(a) shows the drawing primitives 316 to 329. The primitives 316 to 329 
reference z-levels 333, 332 and 334, shown in Fig. 33(c). The z-levels are shown in 
absolute depth order - that is they could have absolute depths 2. 3 and 4 respectively. 
TABLE 2 is a table of which z-levels the drawing primitives reference. For example. 
LINETO 316 is directed down the page, and has the z-level 332 to the left of its drawing 
direction. The rendered result is shown in Fig. 33(c). 

TABLE 2 



Drawing primitive 


Left z-Ievel 


Right z-Ievel 


316 


332 


None 


317 


332 


None 


330 


332 


None 


318 


332 


None 


319 


332 


None 


320 


332 


None 


321 


332 


None 


322 


333 


None 
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323 


333 


None 




324 " 




None 


325 


333 


332 


327 


334 


332 


326 


334 


332 


328 


334 


^332 


329 


334 


332 



The styles of z-level may include, but are not hmited to. a simple color, a Unear 
blend described by one or more colors, a radial blend described by one or more colors or 
a bitmap image. All of these z-level styles also support a transparency (alpha) channel 
The z-levels 333, 332 and 334 in Kg. 33 represent simple color style z-levels. These z- 
levels are used unchanged through much of the pipeline. 

1.5. Stroking 

Drawing primitives can be associated with a pen width. Drawing primitives with 
a pen width are converted into multiple edges (edges have no width). These edges form 
closed filled shapes that represent the pen stroke. See the section titled Transform. 
Moiphing and Stroking for a detailed discussion. 

1.6. Morphing 

Moiphing is also weU known in the art. Morphing can be defined as supplying 
two sets of drawing primitives for a graphic object and a ratio which specifies that the 
gr^hxc object is to be drawn according to an interpolation between the two sets. This is 
also described in more detail in the section titled Morphing. Stroking and Transfonn 
module. 

2. THE DRIVER MODULE 

The driver module 615 will be discussed in terms of the information it handles 
and what information it passes on to the remainder of the rendering pipeUne. The role of 
the driver module 615 is to organize collections of drawing primitives. Drawing 
prmutives are first collected into graphic objects, as described above. 

Graphical objects in turn can be collected into sprites, as described below. These 
sprites can be specified as havmg properties which apply to the whole collection The 
pnmary role of the driver is to allow efficient high-level operations on the sprites without 
complicating the rest of the graphical pipeline. When the Driver Module 615 outputs 
drawmg information for subsequent modules in the gr^hic pipeline, the properties of a 
spnte are appUed to each drawing primitive of each of its graphic objects (for example its 
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transfonnation matrix). This allows subsequent modules to deal with directed edges and 
z-levels only. 

2.1. Sprites 

The Driver Module 615 accepts sprites as part of its input. Sprites are well known 
m the art and in the Driver Module 615 they refer to a primitive which has a 
transformation matrix, a depth and a Ust of graphic objects which exist within the context 
of the spnte. Sprites can contain zero or more graphic objects and zero or more other 
spates. By "contain", it is meant that the transformation matrix of the sprite is applied to 
all of the graphic objects and sprites the sprite in question owns. The concept of a sprite 
contammg other primitives also means that the depth of all graphic objects and sprites it 
contains are "local" to that sprite. Graphic objects do not contain other graphic objects or 
sprites. 

2.1.1 Sprites: transformation matrices 

Transfomiation matrices are weU known in the art. A sprite's transformation 
matrix applies to aU gr^hic objects owned by the sprite: it defines a local space for that 
spnte. m Fig. 17(b) two sprites and two graphic objects are provided and the manner in 
whach they are to be rendered is described by a tree. Sprite 185 contains both sprite 189 
and graphic object 180. That is. the links 188 and 186 represent ownership relationships. 
Spnte 189, m turn contains a second graphic object 182. 

Fig. 17(a) represents the geometry of the transformation matrices which exist in 
this example. A space 179 contains objects within sprite 185. The object 180 is seen 
located within the space 179 whereby coordinates of the object's 180 constituent drawing 
pnmitives refer to the space 179. Sprite 189 in Kg. 17(b), is also located in this 
space 179. A space 181 represents that in which the graphic objects of sprite 189 are 



25 located. 



In this example, sprite 189 has a transform which has a rotation, a translation and 
a scahng. The translation is represented by a dotted line 183, the rotation by an angle 
184. and the scaling by the relative size of the divisions of 179 and 181. the axes used to 
represent the spaces of sprite 185 and sprite 189. respectively. The scaling, in this 
30 example, is the same in botii axes. 

Still referring to Kg. 17(a). the transformation matrices describing the placement 
of gr^hic objects owned by a sprite are concatenated with the transfonnation matrix 
descnbing the placement of ti^t sprite, m tixe example in Kg. 17(a). tixe transfomxation 
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matrix appUed to graphic object 182 is the concatenation of the transfonnation matrices of 
sprite 189 and sprite 185. This resultant transformation matrix for the object 182 is 
appUed to all the drawing primitives the object 182 contains, 
2.1.2. Graphic objects and their deptii 

The depth of a graphic object is local to the sprite which contains the object. This 
is iUustrated by Figs. 31(a) to 31(c). In a rendering tree shown in Fig. 31(a), a node 294 
represents a sprite which contains another sprite 296 and a graphic object 302. The 
ownership relationship is indicated by directed lines 295 and 301 respectively. The 
sprite 296 in turn contains graphic objects 298 and 299 as indicated by ownership 
relationships 297 and 300 respectively. TABLE 4 provides the local depths of all these 
primitives. 

TABL£4 



Label 


Prinaitive 


Local depth 


Absolute depth 


294 


Sprite 


2 


n/a 


~296 


Sprite 


2 


n/a 


298 


Gr^hic Object (circle) 


1 


3 


299 


Gr^hic Object(square) 


2 


4 


302 


Graphic Object (triangle) 


1 


2 


309 


Background z-level 


1 


1 



The concept of local depth can be illustrated by looking at the ^pearance of sub- 
trees in Fig. 31(a). Fig. 31(b) shows the visual ^pearance of the sprite 296 (being 
objects 298 and 299 rendered in isolation). Since object 298 has a smaller depth value 
than object 299. object 298 appears beneath object 299, Fig. 31(c) shows the visual 
appearance of graphic object 302 and sprite 296, Since gr^hic object 302 has the smaller 
depth value, it ^eais beneath 296. Notice that the local depths of the children of the 
sprite 296 are preserved according to the local depth legend 304. 

The node 303 is the root of the ownership tree and represents a sprite containing 
all the topmost sprites. The root 303 always owns the background z-level 309 at its 
lowest local depth (which is also the lowest global depth). Therefore, this background z- 
level underlies all graphic objects. 
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2.2. The display list 

The Ust of aU sprites and graphic objects for a single frame, including their 
ownership relationships, local depths and transforms is termed the display list. Display 
Usts are weU known in the art. The driver module 615 maintains the display Ust in the 
form of a tree. This tree coUects sprites and gr^hic objects in terms of ownership 
relationships, and is ordered in terms of local depths. Fig. 31(a) also illustrates this 
arrangement. For example, the children of sprite 294 are ordered by local depth; the first 
child 302, is at the lowest local depth of 1, and the next child 296 is at a higher local 
depth of 2. 

2.2.1. Frame rendering 

For each frame of an animation, the display list may be modified prior to being 
rendered to the display. The display list is retained between fiaines. 

2.2.2. Graphic objects and z-Ievels 

Just as sprites can reference multiple graphic objects, graphic objects can 
reference multiple z-levels. Each of these z-levels has a depth within the gr^hic object. 
For example, the graphic object Fig. 33(c) requfres three z-levels with depths as shown in 
Fig. 33(b). 

2.2.3. Local depths and absolute depths 

While local depths are used within the Driver Module 615, later modules require 
absolute depths. The Driver Module 615 assigns each z-level an absolute depth. 

Absolute depths for the z-levels of graphic objects are passed from the Driver 
Module 615 to later modules, where they are simply termed depths: it will be understood 
that aU later modules work with z-levels that have been assigned absolute depths. These 
assignments are done on a firame by frame basis, just prior to rendering. This is only done 
if new display objects or sprites have been inserted into the display list or old ones have 
been discarded. 

Given that Fig. 31(a) shows all primitives for a single frame, TABLE 4 above 
shows absolute depths for all primitives in Fig. 31(a). The absolute depths for all z-levels 
for a single frame start at 1, which is assigned to the special background z-level. Z-levels 
are numbered by contiguous integers upwards from the background z-level. Note that 
sprites do not have an absolute depth per se (although they may record information about 
the depths of their descendants). Sprites exist as containers for information (such as a 
placement transform) related to the drawing objects they contain, but are not drawing 
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ob;e«s u. ^ own right I.ter modules in ^ „^ ^ ^ ^ 

Grap^c objects do not exist outside the driver nodule «S either, lie local dep* of 3 
graph-c object is used to calcuWe a„ <*sotat. depfl. of the z-leveis „ferenced by .he 
draivmg pmmtives that make .V the graphic object 

' disn, ?r'"'°'*'^"'°°''"'°'''^'""'«'*^=- Of the 
TZT- ™^ A method Of calculating these absolute depths is 

U^ed m «e 44 Which Shows a display hst tree. A .ot node 440 of the disply Z 
^ owns a background z-level 44, which underlies aU other z-.ev.ls. iJis 4e 

*ackg»und44,has«.l„westpossibleabsolutedepthofl. Nodes 432^3, make u; the 
10 remainder of the display Kst tree. ^ 

d^a-fir. traversal of the fee. shown by dotted line 442. Assigning absohrte depths .0 
me z, v,^ of a graphic object commences with the lowest z-level in that graphic object 
and wh.c .s assi^ed an absohUe depth one higher «^ the highest z^el of ^ 
15 ™y .^receding- graphic object By -^.eding, it is mean, that the 

object was the prevrous one visited in me depth first travel. M «g. 44. the depi L 
tr^ver^ . shown reaching gmphic object 438. To assign an absolute depfl. .0 the lowest 
-level Of the objec. 438. it is assigned an absolute depth one higher than U.e highestt 
level m object 435. the previous graphic object visited 

^ /''°:r'''''^*«'='^-'"^'^-*'^-~^^uenttalabsolu.edn,«.. 

«1 r "'^ wiU recaU that aU spritTan^ 

^aphrc objects have mdependent local depfl. ranges, and so inweaving is notpossible 

5 ."^^r^ ^^'^'^^-^'-'---^epu.of^.etopmostz. 

5 of descendants. Traversal of ttie tree always starts from the ™ot 

nodes ^""''""""^ display list has been changed by the insertion or removal of 
nod«, or efficency. a minimal update of the tree is performed During traversal of the 

XT T ""^'^ ^-^^ node 

whrch has changed. He z-levels of graphic objects earher (in traversal order) than flus 

> ^ ^ ™hd absohrte depths. Prior to r^hing d.e tirs. changed node. LZZ 

*..2^«.isacMevedbyno«ngthetopmos.abso,utedep.hofeachnode.,Vhen^^^^ 
2 Changed node . encountered, absolute depth assigmnents foUow from this no,«, 
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For example, in Kg. 44. if the object 438 was a new node in the display list tree 
and this was the only change since the previous fiame. the allocating absolute depths' 
would start from object 438. 

3. TRANSFORMATION, MORPHING AND STROKING 

As input, a Morph, Tiansfonn and Stroking module 616 of Kg. 56 takes one or 
more ordered set(s) of coordinates from the Driver Module 615 that collectively define 
one or more outlines of drawing objects to be rendered. Each ordered set of coordinates 
may be accompanied by additional parameters, including: 

- morph ratio 

- transformation matrix 

- stroke width 

- reference to a left-hand z-level 

- reference to a right-hand z-level, and 

- reference to a stroke color z-level. 

The morph ratio, transformation matrix and stroke width parameters describe how 
the ordered set of coordinates describing graphic object outlines should be positioned on 
the display. TTie remaining parameters are references to z-levels that indicate the color 
blend or texture for display pixels within the outlines defined by the ordered set of 
coordinates. 

Kg. 46 is a flow diagram showing the processing performed on each ordered set 
of stepped coordinates by the module 616. The purpose of this processing is to convert 
each ordered set of stepped coordinates (eg. see Kg. 16(a))into a corresponding ordered 
set of edges, where each edge of the ordered set of edges is described by at least a start 
coordmate and an end coordinate in render space. THat is, edges have directionality The 
left and nght references to z-levels that edges possess is m terms of this directionality. 
The morph ratio, transformation matrix and stroke width parameters are used to control 
this processing. 

h addition, the module 616 optionally accepts one or more glyph descriptions. 
These descriptions contain the following information: 

- Glyph position, 

- Glyph height and width, 

- Glyph bitmap, 

- Reference (possibly NULL) to an "on" z-level. 
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- Reference (possibly NULL) to an "ofi" z-Ievel, and 

- Glyph descriptions enter the pipeline at stage 470. 
3.1. Morphing 

If the module 616 receives an ordered set of coordinates corresponding to a morph 
object (ie. in Fig. 46, stage 464 = yes), then two versions of each coordinate are received, 
along with a morph ratio pai^eter. One version of each received coordinate is referred 
to as a start version. A second version of each received coordinate is referred to as an end 
version. The module 616 uses the morph ratio parameter to interpolate a single 
mteimediate version of each received coordinate at stage 467. For example, Hg. 35(a) 
and Fig. 35(c) illustrate an ordered set of coordinates representing a morph "start" shape 
and a morph "end" shape respectively. Both ordered sets of coordinates are shown to 
begm and end at an origin 342 in a "logical" coordinate space. The purpose of the morph 
process is to produce an mtemiediate version of the shape as shown in Fig. 35(b) by 
mterpolating between the "start" version of Fig. 35(a) and the "end" version of Fig. 35(c) 
me versions ("start" and "end") of each coordmate are presented as an ordered set of 
coordmate-pairs, in this example 339 and 346 will be a first pair, followed by 343 
and 340, 344 and 347. and a final pair 341 and 342. The morph ratio is used to 
mterpolate an mtermediate version from each coordinate-pair. If a morph ratio of 0 is 
used to represent a "start" shape of Fig. 35(a), and a morph ratio of 1 is used to represent 
an "end" sh^e of Fig. 35(c), then a morph ratio of 0.25 would correspond to the sh^e 
shown m Fig. 35(b). The moiph process would therefore produce, as output, an ordered 
set of coordinates illustrated by 350, 351, 352 and 353. 

3.2. Transformation 

Next, in Fig. 46, the transformation matrix is applied to the coordinates at 
stage 468. The transformation matrix allows rotation, scaling and/or translation of 
coordmates (and therefore drawmg objects) to be specified. In this description, the 
transformation matrix is said to transfomi the coordinates (and therefore the drawing 
objects fliey describe) &om a "logical space" mto a "render space". 

3.3. Generating Edges 

After flie ordered set of coordmates have been moiphed (if appropriate) and 
transformed, a subsequent stage 465 in the flowchart of Fig. 46 converts the ordered set 
of coordmates mto an ordered set of edges. An example of how the stage 465 may be 
accomphshed is given by the flowchart of Fig. 15. The example assumes that coordinates 
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can fonn part of one of ttreo types of drawing primitives, the three types being straight 
lines, quadratic Bezier curves and new drawing positions. The example also assumes 
additional information (e.^., a tag byte) is provided to indicate what type of drawing 
primitive follows, and when the end of the drawing primitive is reached. 
5 First as seen in Fig. IS, a current drawing position is initialized to (0,0) at 

step 140, then the type of the "foUowing" drawing primitive to be next encountered is 
determined at steps 124. 125, 126 and 127. 

If the type indicates that the following coordinate describes a new drawing 
position (step 124), then the coordinate is read at step 128 and used to set a new current 
1 0 drawing position at step 131 . 

If the type indicates that the following coordinate describes a straight edge 
(step 125). then a new straight edge is generated which is given a start coordinate equal to 
the current drawing position at step 129. A coordinate is then read at step 132 and this is 
used for both the new straight edge's end coordmate at step 134 and the new drawing 
15 position at step 136. 

If the type indicates that the following two coordinates describe a (eg. a quadratic 
Bezier) curve (step 126), then a new curve is generated which is given a start coordinate 
equal to the current drawing position at step 130. A first coordinate is then read at 
step 133 and used as a "control point" coordinate of the new curve at step 135. A second 
coordinate is then read at step 137 and used as both the end coordinate of the new curve at 
step 138, and the new drawing position at step 139. 

Processing continues until the end of the ordered set of coordinates is reached at 
step 127. 

3.4. Decomposition of strokes into edges and z-levels 

Returning to Fig. 46, the next step of processing is optionaUy availed at step 471 
and is a stroking step 466. Stroking is the process of generating one or more outlines 
such that they simulate the effect that a pen of given thickness has been used to trace 
along a path of curves and vectors. Example of stroking are iUustrated in Fig. 50(a) to 
Fig. 50(e). Fig. 50(a) shows an ordered set of three edges 523 to be stroked. A generated 
stroked outline 524 is seen in Fig. 50(b) providing the effect that a circular-tipped pen of 
a given pen width 525 has traced the path of the three edges to be stroked. 

Although Fig. 46 suggests that the stroking step 466 must be performed after the 
transformation step 468, it may also be desirable to aUow the possibihty of stroking to 
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take place prior to transformation, producing 6im^ remits. For example, reftiti«, 
again to Pig. 50(a). if an original shape described by the edges S23 is firs, transfcnaed to 
produce U>e shape S2« shown in Hg. 50(c). and then ste.k«J. fte result will be a,e 
*ape 528 as shown in Fig. 50(d). However, if the original shape described by the 
^ges 523 is first stroked 524 and d,en the same tnms&m. is apphed. the result becomes 
the shape 527 shown in 1^ 50(.). n is possible for the TCIE system 699 ,o achieve 
erther result by optionally perfcnning the transformation of coordinates of step 468 ate 
Stroking 466. 

Note that the following discussion of stroking assumes that stroking is performed 
m a coordinate system with a positive Y-axis that is 90 degrees clockwise to a positive X- 
axis (e.g., X mcreases to the right, Y increases downwards). 

Tho module 616 strokes an original ordered set of edges by iterating through the 
edges in the ordered set of edges. For each edge, a new left-hand edge and a new right- 
hand edge is generated corresponding to a left and right extent of a circular-tipped pen 
stroke that has been centred and traced along the path of the original edge. EUiptic arcs 
^e used to generate joins between two successive original edges, as well as end-caps at 
the start and end of an original ordered-set of edges that does not fonn a closed shape An 
ordered-set of edges is only closed if the start coordinate of the first edge of the ordered- 
set of edges is equal to the end coordinate of the last edge of the ordered-set of edges 

The arcs are placed as ciix^ular arcs in object space, and then transformed via 
processes described in later sections to elhptic arcs in display space. Although this 
prmntive is currently only used internally, it should be noted that arcs could readily be 
exposed as drawing primitives available to the user. 
3.4.1. Stroking a straight edge 

Fig. 49(a) shows an example of straight edge 473 to be stroked, described by a 
start coordinate 474 and an end coordinate 472. Kg. 49(b) shows the desirod result of 
strokmg the straight edge 473, the i^t comprising a left-hand edge 477 and a right hand 
edge 478. 

To stroke a straight edge, a left-normal vector 479 is generated which- 

- has a direction that is 90 degrees anti-clockwise to the direction of the straight 
edge, and ^ 

- has a length equal to half the pen width of the stroke. 
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A left-hand stroke edge and a right-hand stroke edge are then calculated using the 
coordinates of the straight edge 473, 474 (Xs, Ys) and 472 CXe. Ye), and the left-nonnal 
vector 479 (X„,Y„). 

For the left-hand stroke edge 477. a start coordinate pc^jeft. Ysjeft) and end 
coordinate pCejeft, Yejeft) are calculated as follows: 

XsJcft=Xs+X„ Ysjeft = Ys+Y„ 

Xejeft =Xe + X„ Yejeft = Ye + Yn 

A right-hand stroked edge 478 is generated by subtracting the normal vector from 
the coordinates of the current edge: 

Xs_right = Xs - X„ Ys_right = Ys - Y„ 

Xe.right = Xe - X„ Ye_right = Y^ - Y„ 

3.4.2 Stroking a curved edge 

Fig. 49(c) shows a curved edge 483 to be stroked. Curved edges (quadratic Bezier 
curves or Quadratic Polynomial Fragments - QPFs) are described by a start point 480. a 
control point 481 and an end point 482. 

Fig. 49(d) shows the two left-normal vectors 487 and 488 that must be generated 
to stroke this edge. Both of the two left-normal vectors have a length equal to half the 
pen width of the stroke. The first of the two left-normal vectors 487 is 90 degrees anti- 
clockwise to the vector that runs from the start point 480 to the control point 481. The 
second of the two left-noimal vectors 488 is 90 degrees anti-clockwise to the vector that 
runs from the control point 481 to the end point 482. 

With reference to the Fig. 49(e), a curved left-hand stroke edge is generated that 
has a start point 493, control point 494 and end point 495. The start point 493 of the left- 
hand stroke edge (Xsjeft . Ysjen) is calculated by translating 480, the start coordinate of the 
curved edge (Xs. Ys), by the first left-normal vector 487 (X„i, Y„i): 

Xsjeft = Xs + Xni Ysjeft = Y, + Y„i 

The end point of the left-hand stroke edge (Xejcft . Yejeft) is calculated by 
translating 482, the end coordinate of the curved edge (Xe, Ye), by the second left-normal 
vector 488 (Xrtz,Y„2): 

Xejeft=Xe + X„2 Yejeft=Ye + Y„2 

The control point 494 of the left-hand stroke edge is calculated using the 
intersection of two straight lines. The first straight Une runs through the left-hand stroke 
edge start point 493, and is parallel to the straight line running from the curve edge start 
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point 480. to the curved edge control point 481 . The second strai^t line runs through the 
left-hand stroke edge end point 495. and is parallel to the straight line running ftom- the 
curve edge end point 482 to the curved edge control point 481. 

A curved right-hand stroke edge is generated similarly, as shown in the example 
Rg. 49(f) wherein a start point 503, control point 504 and an end point 505 are drawn 
ITT' coordinate (X..„.^, Y.h^O and end point coordinate (X..,^, Y. .^0 of the 
nght-hand stroke edge are calculated by: 

Xs_right=Xs - X„, Ys_right= Ys - Y„i 

Xc_right = Xe-X„2 Ye_right = Ye-Y„2 

The control point 504 of the right-hand stroke edge is calculated using the 
mtersection of two straight lines. Il.e Grst straight line runs through the right-hand stroke 
edge start point 503, and is parallel to the straight line ruxming fix>m the curve edge start 
pomt 480. to the curved edge control point 481 . The second straight hne runs through the 
nght-hand stroke edge end point 505, and is parallel to the straight line nmning from the 
curve edge end point 482to the curved edge control point 481 . 

Kg. 49(g) shows the resulting left-hand stroke edge 514 and right-hand stroke 
edge 515 drawn with respect to the various control points. 

Note that the technique described above is an approximation of stroking a curve 

and IS unsuitable for curves such as the curve 520 shown in Rg. 49(h) M ■ 

the tnangle fomxed by the steut point 517. conHol point 518 and end point 519 of the 
curve, the control angle is defined to be the angle subtended by (.e, opposite to) the 
straight-line joining start 517 and end 519 points. If the control angle is less than or equal 
to 120 degrees, then thecurvemay be considered tight-angled. A convenient test is to use 
the magnitude of the dot-product of the two left-normal vectors 521 (^„ Y„0 and 522 

(Xn2, Yn2): 

then curve is tight-angled. 
Methods of bfeeotmg Quadratic Bezier curves described by a start, conttol a»l end 
pom, are lotown in .he art. Curved edges that »re detennmed to be tight-angled are first 
30 btseoted into ^vo equivalent curved edges. ,h«^ bisected again. resuMng in a total of 
four curved edges. ITre outer two new edges (a>ose no, adjacent to ft. original tight 
angle) are tt>en stroked usmg the pn,cess descriied above, lb. in„« hvo new edges are 
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vectorized into a path of ^proximaling strai^t edges using techniques weU known in the 
art. This path is then stroked using the process described in Section 3.4.1. 
3 A3. Stroking a join 

Additional curves need to be generated to stroke the area where edges of a path 
join. A join is defined to have an entering edge and an exiting edge. Fig. 45(a) shows 
two straight edges forming a join 444, where edge 443 is an entering edge since the joia 
forms its end point, and edge 445 is the exiting edge since the join forms its start point 
Left hand stroke edges 446 and 447. and right hand stroke edges 448 and 449 have been 
generated using the process described above. Left-nonnal vectors will have been 
determined for the end of the entering edge 462 and the start of the exiting edge 463. 
Considering the left-hand side first, a test is made to determine if it is necessary to 
generate a curve outhne for the join that comiects the end of the entering edge 453 with 
the start of the exiting edge 454. Given that a left-normal vector 462 with components 
(X„i, Y„0 is akeady known for the end of the entering edge to be stK;ked, and a left- 
normal vector 463 with components Q^, Y^) is already known for the start of the 
exiting edge to be stroked, then the following test can be used to determine the necessity 
of generating the curve outline: 

If Y„iX„2 < YraXni, then a curved edge is used to link the left-hand edges of the 
stroke, and a straight edge is used to link the right-hand edges of the stroke. 

Smularly, if Y„iX„2 > YnzXm, then a curved edge is used to link the right-hand 
edges of the stroke, and a straight edge is used to link the left-hand edges of the stroke. 

If YniX„2 = Y„2X„i then the edges forming the join have either equal or opposite 
direction at the join. The process for such a case is given in the Section 3.4.4 below. 

In the example shown in Fig. 45(a). the above test would indicate that a curved 
edge must be generated for the left-hand side of the join. This curved edge will be an 
exact circular arc with the following properties: 

- the start point and end points of the arc are points 453 and 454; 

- the radius of the arc is the length of the left-hand vector 462; 

- the center of the arc is the original path point 444; and 

- the arc proceeds in a clockwise fashion fi-om start point to end point (the 
arc would proceed counter-clockwise if the right-hand side of the join were 
being processed). 
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These values are used to generate an elliptical arc edge, the further processing of 

which is described below. 

Continuing the example shown in Rg. 45(a), the above test would indicate that a 
straight edge must be generated for the right-hand side of the join, linking the end point of 
a first right-hand edge 451 to the start point of a second right hand edge 450. 

Fig. 45(b) shows a situation where edges 455 and 456 of the same side of 
different stroke paths do not join. A new edge can be created to extend ftom 
corresponding vertices 460 and 461 and which is curved about a control or focal point 
452 associated with the edges 455 and 456. Fig. 45(b) also shows the corresponding 
opposite edges 457b and 457a which do not join. A new edge is inserted between the 
vertices 458 and 459 to provide closure for the path. This is a straight edge as its effect 
on stroke fill is nil. 

3.4.4. Stroking Equal or Opposite Edge Joins 

In the description of the technique of stroking a join above, a test was described 
for mdicating when two original edges forming a join to be stroked had either equal or 
opposite direction: 

where (X„i, Y„0 is the left-nomial vector for the end of an entering edge to be stroked, 
and (X„2, Y„2) is a left-normal vector for the start of an exiting edge to be stroked. 

Further, if = x„j then the lefl-normal vectors are equal and therefore the 
entering and exiting edges have the same direction at the join. No additional edges are 
reqmred for stroking the join. 

If Y„jX^ = Y„^„j but ^Xni then the entering and exiting edges have opposite 
directions at the join, and therefore an end-c^ must be generated for the join, as 
25 described below. 

3.4.5. Generating end-caps at the end of a path 

End-caps are generated for the tenninating points of an unclosed path to be 
stroked. End-caps also have to be generated for joins between edges where the entering 
edge and exiting edge are opposite in direction (as described above). 

The process generates an end-cap for a temunating point using an eUiptic arc. 
An example of an end-cap of a stroke is given in Kg. 18(a). showing an original edge of a 
path to be stroked 192, terminating at a point 193. The left-hand stroke edge 195 and 
right-hand stroke edge 200 are drawn terminating at end points 196 and 198 respectively 
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A left-nonnal vector 194 with components (X„,Y„) for the end point of the terminating 
edge is shovm. 

To generate the end-cap, an eUiptic arc is placed with the foUowing properties: 

- the arc begins at the left-hand end point 196 and terminates at the right- 
5 hand end point 198; 

- the centre of the arc is the original path point 193; 

- the radius of the arc is the length of the left-normal vector 194; and 

- tiie arc moves around the circle clockwise. 

When stroking a path that has a vaUd left z-level reference and/or a vaUd right z- 
10 level reference (in addition to a vaUd stroke z-level reference), then this indicates the 
stroked path also forms part of an enclosed area to be fflled. For such a path, it is 
necessary to add an additional two straight edges to any end-caps. 

In this case, a new left-normal vector 199 with components (X„_taee. Y„_teee) is 
created such that it is equal to the left-normal vector for the end point of "the terminating 
15 edge rotated 90 degrees clockwise. A fcaee coordinate 197 (X^, Yj^^) is then generated 
using the coordinate of the terminating point 193 pCj, Yj) and the new left-normal 
vector 199 with components (K^j^^, Y„ j^^): 

Ximee = Xj + X„_ta,ee , Yknee = Yj + Y„_io,ee. 

Fig. 18(b) depicts the additional two edges, 207a and 207b. which both start at the 
Icnee point 197, and both end at the terminatmg point of the original edge to be 
stroked 193. A first additional edge 207a is an additional left-hand stroked edge, and a 
second additional edge 207b is an additional right-hand stroked edge. A description of 
how z-level references are assigned to left-hand stroked edges and right-hand stroked 
edges is given below. 

25 3.4.6. Endcaps between stroked and unstroked edges 

End caps are also generated when an edge referencing a stroke fill is followed in 
the path by an edge which does not reference a stroke fill. Fig. 28(a) shows a triangular- 
shape that is partially stroked. Vertex 275 is where stroked edge 276 joins non-stroked 
edge 277. An end cap is generated at vertex 275. 

30 This end cap is constructed by the same method as shown in the previous section - 

that is, by the same method used for end caps generated at the end of the path, and 
additional steps as described below. The results of applying this method are shown in 
detail in Fig. 28(b). 
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Consider the interior fiU of the shape, which is the left hand ffll referenced by 
edge 278. From this, it can be seen that the addition of the smaU end cap edge 280 
ensures the interior ffll is completely bounded by a closed set of edges. Edges 279, 280, 
282, 278 are the in the set of edges that enclose the shq,e fiU in the region of the join' 
Additional edges not shown in the diagram enclose the ffll in regions not in the join. It is 
well known in the art that methods of filling shapes rely on those shapes being closed - 
particularly when winding rules and winding comits are used to fill areas bounded by 
edges. 

Edge 278 is the only edge shown in Fig. 28(b) that was not generated by the 
stroking method - it taken from the set of input edges and included m the set of output 
edges. 

3.4.7. Z-level assignments for opaque strokes 

When an ordered set of coordinates are provided as input, and that ordered set of 
coordinates are to form a stroked path, then a vaUd stroke z-level reference and a pen 
width will be required as input in addition to a left z-level reference and a right z-level 
Inference. There is no difference between the format of a z-level that is used for a stroke 
and a z-level that is used to ffll a path. Once the ordered set of coordinates have been 
processed into an ordered set of edges they can be stroked using the technique described 
above. The module 616 produces a set of left-hand stroke edges and a set of right-hand 
stroke edges. Given that the module 616 received a left z-level. right z-level and stroke z- 
level as input, and the stroke z-level is determined to be opaque, then the produced left- 
hand stroke edges are assigned: 

- a left z-level reference that references the left z-level received as input; 

- a right z-level reference that references the stroke z-level received as 
input; 

and the produced right-hand stroke edges are assigned: 

- a left z-level reference that references the stroke z-level received as 
input; 

- a right z-level reference that references the right z-level received as 
input. 

In this section, a reference to a left or right stroke edge is a reference to an edge on 
the outer boundary of a stroked edge as seen following the direction of the edge. In this 
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regard, a left stroke edge will appear on the right side of the edge when the edge is 
directed from a top-left of the display to the bottom-right of the display. 

When this 2-level assignment is used, flie original edges of the stroked path are 
discarded and therefore completely replaced by the produced left-hand stroke edges and 
right-hand stroke edges. 

An example is shown in Kgs. 51(a) to 51(e). Elg. 51(a) shows a set of input 
edges for the stroking method given above, and which define a closed path. Edges 533 
532 and 534 are aU associated with left z-level 530. right z-level 531 and stroke z- 
level 529. In the case of an opaque stroke, the edges output from the stroking method for 
mput edge 533 are shown in Hg. 51(d). together with their z-level assigmnents The 
visual result of stroking the path given in Fig. 51(a) is shown in Kg. 51(b). 

Note that the produced left hand stroke edge 535 has a left z-level reference 538 
that references the left z-level 530 provided as input, and has a right z-level reference 539 
that references the stroke z-level 529 provided as input. Also, the produced right hand 
stroke edge 536 has a left z-level reference 540 that references the stroke z-level 529 
provided as input, and a right z-level reference 541 that references the right z-level 531 
provided as input. The original stoke edge 533 is discarded. 
3.4.8. Z-Ievel assignments for transparent strokes 

An alternative z-level assigmnent to the above is now given. The assigmnent is 
particularly suited to the stroking of an ordered set of edges that has a stroke z-level 
havmg some degree of transparency. Jn the z-level assigmnent. the produced left-hand 
stroke edges are assigned: 

- a null left z-level reference, 

- a right z-level reference that references the stroke z-level received as 
input; 

and the produced right-hand stroke edges are assigned: 

- a left z-level reference that references the stroke z-level received as 
input, 

- a ««// right z-level reference. 

When this z-level assigmnent is used, the original edges are not discarded - the 
output of the stroking process is the union of the set of original edges, the set of produced 
left-hand stroke edges and the set of produced right-hand stroke edges. The original 
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edges are assigned a left z-level refei^ce provided as the input left z-level reference and 
a nght 2-level reference provided as the input right z-level reference. 

Fig. 51(a) shows a set of input edges for the stroking method given above. 
Edges 533, 532 and 534 are all associated with left z-level 530. right z-level 531 and 
stroke z-level 529. In the case of a transparent stroke, the edges output from the stroking 
method for input edge 533 are shown in Fig. 51(e), together with their z-level 
assigmnents. The visual result of stroking the path given in Fig, 51(a) is shown in 
Kg. 51(c). 

Note that the produced left hand edge 542 has a left z-level reference 545 that is 
null, and has a right z-level reference 546 that references the stroke z-level 529 provided 
as input. The original edge 533 is not discarded, and has a left z-level reference 550 
which references the left z-level 530 provided as input, and a ri^t z-level reference 549 
which references the right z-level 531 provided as input. Also, the produced right hand 
edge 543 has a left z-level reference 547 that references the stroke z-level 529 provided as 
ii^ut, and a right z-level reference 548 that references that is null. 

Kg. 63 illustrates a path 6300 which is desired to be stroked and which is formed 
by a number of directed edges 6302, 6304, 6306 and 6308. As seen, the directed edges 
extend from and join by virtue of vertices 6310, 6312, 6314, 6316 and 6318. 

In this example, it is desired for the path 6300 to be stroked according to stroke 
widths associated with each of the directed edges 6302-6308. In this regard, it is noted 
that the directed edges 6304 and 6306 have the same stroke width and color and as such, 
those two edges collectively define a discrete portion of the original path 6300 which is to 
be stroked. 

As indicated previously with respect to Rg. 18(a) and Rg. 18(b), artifacts can 
occur when stroking the end of an edge. Specifically, as illustrated in Fig. 63, where 
edges having different stroke widths intersect (eg: at the vertices 6312 and 6316), it is 
essential that accurate stroking take place for all edges so that accurate compositing may 
be performed from the edge information. In this regard, the different stroke widths may 
result in one stroke being composited over the other and such can provide substantial 
errors where opaque strokes are used. Further, less substantial but stiU significant errors 
may occur when transparent strokes are used. In order to ensure that the path 6300 is 
accurately stroked it is necessary to ensure that the individual stroke paths formed by the 
discrete edge portions of the path 6300 are accurately reproduced. 
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This is initially performed by identi^g those vertices of the original stroke 
path 6300 at which the stroke width changes. From Kg. 63. it is seen that stroke widths 
change at the vertices 6312 and 6316. At aU other vertices, the stroke width is constant 
and corresponds to that of the corresponding stn>ke path to be created. Further 
Identifying the vertices where the stroke width changes, allows for those other vertices 
where there is no change (eg: Ihe vertex 6314) to be accurately stroked. With the vertices 
^propriately identified, separate stroke paths can then be created for each of the discrete 
portions of the original path 6300. For a first discrete portion, defined by the edge 6302 
stroke paths 6320 and 6322 are each originating from the vertex 6312 and being directed 
toward an apex located on an extension 6338 of the directed edge 6302 and extending 
from the vertex 6310. The stroking of the paths 6320 and 6322 about the vertex 6310 
may be performed in the fashion shown previously described in Fig. 18(a). The stroking 
of the paths 6320 and 6322 about the vertex 6312 is performed in the feshion previously 
described with reference to Kg. 18(b) and it is noted that Kg. 63 exaggerates the paths 
6320 and 6322 extending from the vertex 6312 (cf. the edges 207a and 207b seen in Kg 
18(b)). Notably, the stroke paths 6320 and 6322 coUectively envelop the corresponding 
discrete portion (formed by the edge 6302) of the original path 6300. 

A similar approach is performed for the stroke path associated with the 
edges 6304 and 6306. In this particular case, each of the ends of the path defined by the 
vertices 6312 and 6316 incorporate arrangements corresponding to that previously 
described in Kg. 18(b). Stroke paths 6328 and 6330 associated with the edge 6308 are 
created in a similar fashion to those for the edge 6302. Significantly, in each of these 
examples it is necessary for the stroke paths to transit the vertices identified as being at a 
point on the path where the stroke width changes. 

An extension of the arrangement of Kg. 63 can now be understood with reference 
to Kgs. 51(b) or 51(c). where a closed path is shown. The arrangement of Kg. 63 
enables a closed path incorporating discrete portions to be stroked such that an interior of 
the path can be accurately filled using the negative winding rule and the fill rules 
previously described with reference to Kgs. Sl(a)-51(e). m this fashion, the previously 
described method of replacing edges of the original path with edges defining the stroke 
path having left and right fill can be used to define the interior of the closed path and the 
consequential fiU levels for the stroked portions thereof and the interior. 
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3.4.9. Transformation of Stroking Primitives 

If stroking is perfonned in render space, then the lines. Bezier curves and arcs 
produced by the stroking module do not need to be transfonned. However, if stroking is 
perfonned in object space, then these primitives must midergo the same transformation as 
the original edges of the sitape. 

Stroke lines and Bezier lines are transformed in a similar manner to shape lines 
and Bezier curves, as discussed in Section 3.2. Stroke end-caps and joins, on the other 
hand, must undergo a separate procedure, as the result of applying an affine 
transformation to a circle arc is a generalized eUiptic arc. 

A generalized elUptic arc is shown in Fig. 19. The underlying ellipse 211 can be 
described by a major axis 208.. a minor axis 209. a center point 215. and an angle of 
rotation 210. The eUiptic arc fragmeat 212 that lies on this eUipse is fully described by 
the further provision of start Y coordinate 213, end Y coordinate 214. and fragment 
direction (clockwise or counter-clockwise). In the case of ellipse fragment 212, the 
1 5 direction is counter-clockwise. 

There are several approaches towards retrieving eUiptic coordinates from an 
underlying circle and a transformation. One approach is to find the two solutions mod ir 
of the following equation: 

tan(2^) = . 2a6 + 2ct/ 
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a'-b'+c^-d' 



20 where T = ^ j is the non-translating part of the transformation matrix. 

The normalized major and minor axis vectors (relative to the center of the elUpse 
and for a unit circle) are then given by: 



fcos0^ 
sind , 



for the two solutions of ^. One of these normalized axes (or its inverse) A' will he in the 
first quadrat. The length of this axis vector multipUed by the radius of the pre- 
transformed circle gives the length of the major axis of the post-transformed elUpse. The 
angle of this axis vector from horizontal gives the rotation of the ellipse. The length of 
the other axis vector multipUed by the radius of the circle gives the length of the minor 
axis of the ellipse. 
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The center of the ellipse is found by ^plying the user-suppUed transfonnation to 
the center of the circle. The start and end coordinates of the eUiptic arc are found by 
applying this same transformation to the start and end coordinates of the original circle. 

It is possible that the user-suppHed transformation is inverting in one dimension. 
If this is the case, then the direction of the arc fragment must be changed. To test whether 
the transformation is inverting, it is sufficient to take vectors 



apply r to them, and determine the rotation from horizontal of the resulting vectors. The 
two vectors will inscribe an angle that is less than ISC'*. If 



is on the counter-clockwise side of this angle, then the transformation is an inverting one; 
else it is not. 

3.5. Filtering 

Having generated coordinates that describe an edge in terms of a start coordinate 
(Xs, Ys), end coordinate (Xe, Ye), and for curves only, an additional control coordinate 
(Xc, Ye), the next process is to discard edges that clearly do not affect flie output frame, 
this being a filtering process 470 seen in Fig. 46. At this stage in the processing, all the 
coordinates of an edge are in "render space" (since they have been transformed) and 
therefore can be directly compared with the bounding coordinates of the display. For 
example, if the frame has a width of w and a height of h, the top-left of the frame 
described by (0,0) and the bottom-right of the frame described by (w - I.h - 7), then a 
straight edge can safely be filtered (i.e., discarded) if: 

(Xs >= w) AND (Xe >= w) //off to the right 

OR 

(Ys<0)AND(Ye<0) //off the top 
OR 

(Ys >= h) AND (Ye >= h) // off the bottom 
For the purposes of filtering, glyphs are treated as if they were straight edges that 
stretched from the top-left comer to the bottom-right comer of the glyph. 

A similar test for curves (which have the additional control coordinate) is: 
PQ>=w)AND(Xc>=w)AND(Xe>=w) //offto the right 
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OR 

(Ys<0)AND(Yc<0)AND(Ye<0) //offthetop 
OR 

(Ys >= h) AND (Yc >= h) AND (Ye >= h) // off the bottom 
Straight edges that are horizontal (ie. Ys = Ye) are also filtered. 
It should be noted that edges which are completely off to the left of the display are 
not discarded - such edges do affect the display output. 
3.6. Generating edge-tracking parameters 

In Fig. 46, the purpose of the following process 469 is to convert the edges into a 
modified representation, which may include the calculation of tracking parameters. This 
modified representation is more suitable for processing by subsequent modules, and the 
modified representation contains at least: 

- a start coordinate PC Y), 

- a reference to a left z-level and/or a reference to a right z-level, 

- an end Y ordinate, and 

- tracking parameters. 

The term 'tracking parameters" is used herein to describe one or more parameters 
that allow the X-ordinate of an edge to be calculated incrementally with respect to unit 
increments of Y, unless contrary intentions are indicated. For example, a straight line can 
be described by a start coordinate (Xs, Ys), an end Y-ordinate Y^, and a delta-X term that 
represents a change in X-ordinate corresponding to a unit step change in Y-ordinate {i.e., 
a gradient). The delta-X term would be a single tracking parameter for the edge. 

Because the TCIE system 699 always renders from top left to bottom right, edges 
(straight-lines or curves) are required to be specified such that they have an end Y- 
ordinate that is greater than the start Y-ordinate. This can be ensured by comparing the 
start and end Y-ordinates. and swapping the start and end coordinates if appropriate. If 
the start and end coordinates are swiped, then the left and right z-level references must 
also be swiped. 

3.6.1. Generating straight-edge tracldng parameters 

It has akeady been indicated that a straight Une can be described using a delta-X 
term as a tracking parameter. However, delta-X will often be required to contain 
fractional data. As an alternative to representing delta-X using floating-point or fixed- 



wo 2004/114223 



-36- 



PCT/AU2004/000842 



point format, a more accurate representation is to store an integer part and a remainder 
part for the gradient. 

In a software implementation, this process may be implemented in C-language 
code to calculate the two parts of the gradiait: 

where PQ. Ys) are the start coordinates of the edge, and (Xe, Ye) are the end coordinates 
of the edge. 

For the edge to be accurately described in a suitable format, the Grad and lent 
terms above are provided, along with a start coordinate (Xs, Y3), an end Y-ordinate (Ye), 
left and right z-level references, and additionaUy, a left flag (Boolean value), where: 
Left flag =TRUE if(Xe<X^ 

= FALSE if(Xe>=X^ 
3.6.2. Generating quadratic Bezier curve tracking parameters 

Step 465 of Kg. 46, as described above generates, for each curve: 

- a start coordinate, 

- a control coordinate, and 

- an end coordinate. 

Because the coordinates of the curve have already been transformed (during 
processing of the corresponding ordered set of coordinates in step 468), the coordinates of 
the curve relate to positions in "render space" Fig. 7 shows an example of a Bezier curve 
represented by a start coordinate 58, a control coordinate 59 and an end coordinate 60 in a 
render space indicated by an origin (representing the top-left of a frame of animation 61) 
and X and Y axes (63 and 62 respectively). The process of generating tracking 
parameters for quadratic Bezier curves from a start coordinate 58 (X,, Y^). control 
coordinate 59 (Xc, Ye) and end coordinate 60 (Xe, Ye) is now described with respect to 
Fig. 41. 

In Fig. 41, a first step 403 tests for the case where the start, control and end 
coordinates are co-linear. If this is the case then the control coordinate is ignored and 
tracking parameters for the edge are calculated as described for straight-edges in step 404. 

A next step 405 checks the quadratic Bezier curve to see if it is monotonic in Y 
(that is to say that for each Y ordinate, there is only one corresponding X ordinate through 
which the curve passes). For example, the Bezier curve in Fig. 7 is clearly non- 
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monotonic in Y. It is required that such non-monotonic quadratic Bezier curves be 
described using two equivalent curved edges that are monotonic in Y. The foUowing 
logic can be used to determine whether or not a quadratic Bezier curve defined by the 
start point (X.. Y.), control point (Xc, Yc) and end point (Xe. Y.) requires two curved 
5 edges: 

Let signO be a function that returns a Boolean value TRUE if the operand is 
positive (and FALSE if the operand is negative) and if: 
(sign(Ys - = sign(Y, - 2Yc + Y^) 
AND 

10 i%-Y^<\Y,-2Yc + Ye]^ 

then the Bezier requires two curved edges. 

In the case where it is determined that two curved edges are required, a point 
(Xspiit. Yspiit) on the origmal non-monotonic Bezier is calculated using the formulae: 



- - + 2XXY, - YMY - r.) + X.(Y^-Y\^ 



^ split — 



Y,-2Y,^Y, 



(Y,-2Y^+Y,y 



Two monotonic curved edges that represent the original non-monotonic quadratic 
B^ier are then created in step 409 using: 

- a first curved edge with: 

- start coordinate = start coordinate (Xs, Ys) of original non-monotonic 

quadratic Bezier, and 

- end coordinate = (X^,it, Ysp„0; and 

- a second curved edge with: 

- start coordinate = (X^pMu Ysp„0; and 

- end coordinate = end coordinate (Xe, Y^) of original non-monotonic 

quadratic Bezier. 

It has akeady been indicated that edges (straight-lines or curves) are required to be 
specified such that Ihey have an end Y-ordinate that is greater than the start Y-ordinate. 
This can now be ensured by comparing the start and end Y-ordinates of each curve in 
step 406, and swapping the start and end coordinates if appropriate in step 410. If the 
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start and end coordinates are sw^ed, then the left and right z-level references must also 

be swapped. 

The next step 407 operates to "cUp» the curve if part of the curve is above the 
frame being rendered (Le., it has a start Y-ordinate that is less than 0. where Y = 0 
rq,resents the top of the frame). A top-most ordinate Y,op is detennined by: 

Ytop=max(0, Ys) 

where max(a,b) is a function that returns the largest of the two operands a or b. 

The next test detennines whether or not, for each curve, it is necessary to describe 
that curve as a quadratic polynomial fragment (QPF). This is necessary to avoid a divide 
by zero in solving subsequent quadratic equations. A quick test for this situation based on 
the start, end and control points, is performed in step 413 as given below: 

Ys + Ye = 2Yc , 
implies that ttie curve is a QPF. 

If it is determined that the curve is a QPF. then three tracking . parameters A, C and 
D are generated in st^ 411. These are derived &om the start, end and control points of 
the curve, and the top-most coordinate Ytop, as follows: 

Gttven intennediate values: 

iK-Y,r 

^ . (4XX-2XX-2X.Y.) 

(ye-Y,y 

p - (^,+^.-2X) 
iYe-Y,)' 

A, C and D are then derived thus: 

^ = 4 + %,+ AC 

C = C,+A(24,+1) 
D = 2D, 

If it is determined in step 413 that the curve is not a QPF, then tracking parameters 
A, B, C and D are generated in step 408. A, B, C and D are derived from the start, end. 
control coordinates, and the top-most ordinate Ytop. by the following calculations: 
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Let: 



Then: 



Xg Xs ^Xe 
Xh — Xc^Xe 
Yterm ~ ^ 2Yji 

mix=XgYH~XhYg 



•^term 



in J 2inixxY^ 



term 



Y * 

3.6.3. Determiniiig the sign 

The parameters A, B, C, D are used by a later process to track the X-ordinate of a 
1 5 quadratic Braier curve using an equation of the form: 

The last step 412 of Fig. 41 in preparing the tracking parameters is to determine if 
the square-root temi should be added or subtracted from A. The following pseudo-code 
can be used based on A. X^, Ys, Xe, Ye and D, where the result "sign = +1" indicates 
20 addition of square-root term and "sign = -1" indicates subtraction of square-root tenn: 

if (A>0)sign = -1; 
else rf (A <0) signs +1; 
else 
{ 

25 if(Ys<Ye)sign = -1; 

else slgn = +1; 



if (D<0) sign = -sign; 
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If (Xs < Xe) sign = -sign; 

} 

3.6.4. Generating Elliptic arc tracking parameters 

Step 466 (described above with respect to Kg. 46) generates, for each arc: 

- Ellipse major and minor axis radii *a' and *b'; 

- Ellipse angle (of axis a) &om horizontal '9'; 

- Ellipse center (x, y); 

- Arc start and end coordinates (x^, ys) and (xe, ye); and 

- Left and right Z-levels references, 
EUipses can be described as circles skewed only with respect to Y and scaled only 

with respect to X. Hence, to shnpUfy eUipse-tracking, the TCIE system 699 treats all 
eUipses as transfonned circles. 

The first step that the calculation of tracking parameters 469 must take when 
converting elUpse edges is to determine the circle that underUes the elUpse. Kg. 20 
demonstrates this process. Ellipse 216 is related to rectilmear eUipse 217 by a skew 
factor. The magnitude of this skew is the gradient of Ime 218, which is not an axis of 
216, but is instead a hne firom the lowest extent of the elhpse to the highest extent. 

The height of this hne (firom lowest point to center) can be calculated by the 
formula: 

The width of this hne (again firom lowest point to center) is given by: 

h 

The skew 'e' is therefore: — . 

h 

ElUpse 217 is then converted into a circle 219, with height h. This is simply 
25 achieved by applymg a scale factor: 

ab 

to the eUipse 217. 

For elhpses which are much wider than they are long (i.e. for which a » b and i? 
is small), transforming the generated circle does not give sufficient resolution, and the 
30 resulting eUipse appears blocky. This problem is solved by supersampUng - circles of a 



wo 2004/114223 



-41- 



PCT/AU2004/000842 



much larger radim are tacked, aid results tom several linea are summed together be&re 

transformation. 

A conservative q,pioach for detennining the size of the circle for supersampling 
IS to contmue doubling a scale factor (initialized to 1) while a > 2 * scale * b. This 
^pioach assumes that a >= b and -90 < 0 < 90, and is only necessary if -45 < 0 < 45. 

As described below, circles are tracked using a modification of Bresenham«s 
cnrcle algorithm. Because of the nature of real-time object rendering, only fragments 
which monotonically increase in Y are handled. Hence, the second step performed by 
st^ 469 of Fig. 46 is to split arc fragments that traverse the upper or lower apex of an 
ellipse into two or more fragments. 

One possible implementation of this spUt is described here. First, the eUiptic start 
and end coordinates are converted into cfrcular start and end coordinates by applying 
skew 'e' and scale T. Next, the following algorithm is executed. This algorithm accepts 
start and end points (S) and (B) and the direction (d) (clockwise or counter-clockwise) 
Further mputs are the topmost (T) and bottommost (B) points of the circle, given by 
(Xc, yo-h) and (x^. y,+h) respectively. Tie algorithm produces a Ust of fragments that 
consist of a start y-oidinate. an end y-ordinate. and a circle 'side' (left or right). 

Considering Fig. 67, the x-ordinate of S and E are compared in step 6701 to the x- 
ordinate of the circle's centre (x^, ye). If S and E he on opposite sides, then a test is made 
at step 6702 to see if the top or the bottom of the circle is crossed. The test is: does S He • 
on the left and d is clockwise or does S Ue on the right and d is counter-clockwise? If 
true, the top of the circle is crossed and the output will be as indicated at 6703 (T->S T- 
->E). Circle side follows from the side of the lowest point in the segment, so for the lase 
shown in 6704, the output will be (T->S(L), T~>E(R)). m Fig. 67. 6704 and 6705 are 
the possible arrangements, related by symmetry. In the remainder of this discussion, it is 
understood that every output will inherit the side of the lowest point in that segment. If at 
step 6702 the test shows that the bottom of the circle was crossed, the output is indicated 
at 6706 and the possible arrangements are seen at 6707 and 6708. If S and E were found 
to be on the same side of the circle in step 6701. if E is above S as determined at step 

6709. then the endpoints and direction of the segment are swapped as indicated at step 

6710. thereby simpUfying fturther considerations. 

Continuing with the case of S and E on the same side of the circle, it is determined 
at step 6711 if S and E cross both T and B or neither. The condition is: do S and E Ue on 
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the left with d clockwise, or do S and E Ue on the ri^t with d counter clockwise. If true 
as indicated at 6712, the ou^ut is (S->E) and the possible arrangements are shown ai 
6713 and 6714. If false, as indicated at step 6715 the output is (T~>B. T->S, E->B). In 
this case, the T->B segment will lie on the opposite side to S and E. The possible 
arrangements are shown at 6716 and 6717. 
3.6.5. Generating Glyph edge tracking parameters 

Glyphs are treated as specialized edges by lower parts of the TCIE system 
pipeline 699. Hence, the Stroking Moiphing and Transformation module 616 has to 
convert a glyph description into an edge description. 

The TCffi system 699 tracks glyphs by their right edge rather than their left edge. 
Hence, if (S^, Sy) is the top-left coordinate of the glyph as provided to the Stroke Moiph 
& Transform module 616, W is the width of the glyph, and H is the height of the glyph: 

- the start coordinate of the glyph edge is (S^+W - 1. Sy); 

- the reference to the left z-level is set to be the "in" z-level; 

- the reference to the right z-level is set to be the "ouf * z-level; and 

- the end Y ordinate is Sy + H. 

The glyph-specific tracking parameters consist of the bitmap representing the 
glyph, and the width of the glyph. 
4. SORTING 

The set of edges produced by the moiphing, stroking and transfonn module 616 
are supplied to the sorting module 617 in Fig. 56. The sorting module 617 reorders these 
edges so that they are ordered primarily with ascending values in the Y axis and 
secondarily with ascending values in the X axis (that is. the edges are strictly sorted in Y; 
within each subset of edges that share a Y value they are sorted in X). It is necessary for 
the edges to be ordered in this mamier so that later stages of the rendering process can be 
carried out •'pixel sequentiaUy". Pixel sequentially means that rendering is performed one 
pixel at a time, starting with the pixel at the top left of the screen and proceeding fiom left 
to right across each line of pixels before stepping down to the next line of pixels - the last 
pixel processed is the pixel at the bottom right of the screen. Pixel sequential rendering 
removes the need for the random access frame store used by conventional rendering 
systems, reducing memory usage and typically increasing rendering speed. It should be 
apparent to one skiUed it the art that the convention used herein regardmg the direction 
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and precedence of the X and Y axes can be exchanged for any alternate (but orthogonal) 
convention without detriment to the operation of the described system 699. 

The sorting module 617 achieves its function by the use of the well known radix 
sort algorithm. The radix sort algorithm sorts a set of elements by perfonning a sequence 
of partial sorts on them. Each partial sort is perfonned by sorting with respect to only a 
portion of the sorting index for each element (the portion of the sorting index used at each 
stage has a length in bits equal to the logarithm base two of the radix). This portion of the 
sorting index is used to select which sorting bin to place the element in. The nmnber of 
sorting bins required is equal to the number of the radix (that is. the number of sorting 
bins required is equal to the total number of possible values expressible by each portion of 
the sorting index). As elements are placed in each sorting bin their pre-existing order is 
maintained within each sorting bin. whilst being broken across bins - this ensures that 
subsequent partial sorts do not mido the useful sorting work performed by earKer partial 
sorts. At the conclusion of each partial sort the contents of all bins are concatenated 
together in order, restoring the set of elements being sorted to a (new) simple sequence. 
The first partial sort worics with the least significant bits of the sorting indices, whilst 
subsequent partial sorts work with successively more significant portions of the sorting 
indices. 

The radix sort algorithm is exemplified by Figs. 47(a) to Fig. 47(f). 

Fig. 47(a) shows a sequence of ten elements requiring sorting. Each element has 
a two digit octal sorting index. Note that the octal digits «0. 1, 2. 3. 4, 5, 6, 7" are 
replaced with the symbols "A. B. C. D. E. F. G. H" to avoid confus'ion vJith'the figure 
numbers. 

Fig. 47(b) shows the appUcation of the first stage of a radix eight sort to the 
sequence of elements described in Fig. 47(a). Eight sort bms are used, and elements are 
placed in these bins accordmg to the first portion of their sort indices. The portion of the 
sort indices used for this first partial sort is the least significant three bits (recall that a 
radix R sort works with portions of the sorting mdices with length in bits equal to the 
logarithm base two of R; in this case three bits). The least significant three bits of a sort 
index corresponds to its least significant octal digit, so the first partial sort can be 
described as a traversal of the sequence of elements requiring sorting, apportioning each 
element to the sorting bin corresponding to the rightmost octal digit of that element's 
sorting index. 
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Fig. 47(c) Shows the second stage of the sort. In this stage the contents of aU eight 
sorting bins are concatenated starting with the contents of the first bm and proceeding 
sequentiaUy until the last bin. This results in a new sequence of the ten elements 
requiring sorting. Note that this resultant sequence is correctly ordered with respect to the 
rightmost octal digit; it is not correctiy ordered with respect to the most significant octal 
digit. 

Fig. 47(d) shows the thkd stage of the sort. This stage is a repetition of the first 
stage except that the input sequence is the sequence described by Fig. 47(c) and the 
portion of the sort indices used is the second tbree bits, namely the left hand octal digit 
(most significant octal digit). As this portion of the sort indices uxcludes the most 
significant bit this partial sorting stage is the last one required. This stage can be 
described as a traversal of the sequence of elements requiring sorting, apportioning each 
element to the sorting bin coirespondmg to the leftmost octal digit of that element's 
sorting index. 

Fig. 47(e) shows the fourth stage of the sort. This stage is a repetition of the 
second stage except that the contents of the sorting bins have changed, now being as 
depicted in Fig. 47(d). The contents of all eight sorting bins are again concatenated, 
startmg with the contents of the first bm and proceeding sequentially until the last bin! 
This results in another new sequence of the ten elements requiring sorting. This resultant 
sequence is correctly ordered with respect to the leftmost (most significant) octal digit, 
but not correctly ordered with respect to the rightmost (least significant) octal digit. Note,' 
however, that within subsequences of this new sequence that share identical leftmost octal 
digits the order of the rightmost octal digits is correct. Consequently, this new sequence 
is coirectly ordered according to the sorting indices. The sequence is redrawn in 
Fig. 47(f) for clarity. 

Figs. 47(a) to Mg. 47(f) exempHfy a radix eight sort of a sequence of ten elements 
with sort indices of six bits each. Extension of the principles of tiie radix sort algorithm 
to other integral power of two radices, other sort index sizes and other sequence lengths 
should be apparent to one skilled in the art. Note that larger radix sizes reduce the 
number of partial sorts required (the nmnber of partial sorts required equals the bit size of 
the sort indices divided by the logarithm base two of the radix, rounded up). Smaller 
radix sizes reduce the number of sortmg bins required (the number of sorting bins 
reqmred is equal to the number of the radix). The choice of radix size for a particular 
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sorting task involves a trade off between sorting speed and the memory consumed by the 
sorting bins, so the optimal radix size will vary &om operating environment to operating 
environment 

In ^plying the radix sort algorithm to the graphics rendering system being 
5 described, the sorting indices used are made by concatenating the Y and X coordinates of 
the starting point of the edges. Concatenation of the X ordinate after the Y ordinate 
ensures that the sorting operation sorts primarily in Y and only secondarily in X. After 
the edges are thus sorted, the desired pixel sequential rendering can readily be carried out. 
hi order to accommodate anti-aliasing, the rendering system 610 may describe 
10 edge coordinates with sub-pixel accuracy. Anti-aUasing is described in detail elsewhere 
in this document, but is mentioned here because the representation of edge coordinates 
affects the sorting operation. Sorting is performed to faciUtate pixel sequential rendering, 
so sub-pixel detail of edge coordinates is discarded when creating sorting indices, hi the 
case that coordinates are represented with a fixed-point representation wherem sub-pixel 
accuracy is contamed within the fi:actional digits, this is readily achieved by right shifting 
those coordinates by a number of digits equal to the number of fiactional digits. 

The operation of the radix sort algorithm can be somewhat accelerated by a simple 
optimization. This optimization involves partially overlapping the operation of the 
sorting module 617 with the operation of the previous module (the Transform, Morph and 
20 Stroking module 616). In an un-optimized system that strictly separates the operation of 
these modules, the interaction of the modules can be described as follows. The 
Transforai, Morph and Stroking module 616 creates a set of edges. These edges are 
created in whatever order is convenient for the Transform, Morph and Strokmg 
module 616 and concatenated to a sequence in that order. When this sequence of edges is 
25 complete it is suppHed to the sortmg module 617 en masse for appUcation of the radix 
sort. A system possessing the abovementioned optimization modifies the interaction of 
the modules as described in the foUowing text. The Transform, Morph and Strokmg 
module 616 produces edges as previously described, but does not concatenate them. 
Instead of concatenating the edges they are supplied directly to the sorting module 617 
30 which processes them by appUcation of the first partial sort (that is, the edges are 
distributed to sortmg bins according to their least significant bits). The subsequent partial 
sort operations are performed after the Transform, Moiph and Stroking module 616 
finishes supplying edges to the sorting module 617. This optimization is possible because 
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the first partial sort of a radix sort may be perfonned "on the fly" as a set of objects 
requiring sorting is being developed, whilst the subsequent partial sorts can only be 
performed after the set of objects is finalized. 
5. EDGE PROCESSING 

As can be seen in Fig. 56. the output of the Display List Compiler 608 is an 
ordered set of edges describing the desired output (for a single frame) on the screen. The 
Rendering Engine 610 takes these ordered set of edges as input. Note that a double 
buffer 609 can be used to allow the Rendering Engine 610 to process edges for one frame 
whilst the Display List Compiler 608 prepares edges for a successive frame. 
5,1. Input and output 

The flow of display data through the rendering engine 610 begins with operation 
of an Edge Processing Module 614. whose data flow parameters are smmnaiized in 
Figs. 22(a) and 22(b). A current frame in an animated movie is described by an ordered 
set of new edges 232 and an ordered set of static edges 233. The ordered set of new 
edges 232 comprises those edges that were generated by the Display List Compiler 608 
(described previously) for the current frame. These new edges 232 describe display 
objects that were either not present on the previous frame, or have been moved to a new 
position since the previous frame. The ordered set of static edges 233 describes objects 
that have persisted (and not moved) since they were introduced as new edges on a 
previous frame. 

After Edge Processing 242 has processed all the edges (both new 232 and 
static 233) for a frame, Edge Processing 242 wiU have generated, as output, an ordered set 
of crossing messages 243 provided as input to the Z-level Activation Module 613. and an 
ordered set of static edges 241 which will be used by Edge Processing 239 for the next 
frame (i.e. the set 241 becomes the set 233 at the start of the next frame). 

Edge Processing 242 is also responsible for discarding edges that will no longer 
contribute to the output display after the current frame. Such edges must be marked for 
removal. This removal mark is placed by an external entity (for example, a driver) and is 
outside the scope of this document. As each edge is retrieved from either the ordered set 
of new edges or the ordered set of static edges, the Edge Processing 242 checks to see if 
that edge has been marked for removal. If so. the edge is not transferred to the ordered 
set of static edges for the next frame 241 . 
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5.2. Top-level Operation 

A summary of Edge Processing 242 for a single frame is given in Fig. 24. For 
each frame to be rendered. Edge Processing 242 operates by iterating from scan line to 
scan line (row to row) down the display, and calculating the position at which any edge in 
the ordered set of new edges or the ordered set of static edges intersects the current scan 
line. To one skiUed in the art, the fundamental algorithm is appUcable to other scan 
directions. 

Referring to Fig. 24, at the start of processing a frame, a current scan hne counter 
is initialized to 0 (e.g. the top-most scan line) at step 255. If any edges intersect the 
current scan line, as determined at step 256, then those edges are processed at step 257. If 
the current scan line counter indicates that the final scan line has been processed (step 258 
= yes) then processing is finished for the fiame, otherwise the current scan line counter is 
incremented at step 259 and processing of edges for subsequent scan lines is performed. 

5.3. Active Edge Tracking 

Fig. 22(b) gives an overview of the flow of data through Edge Processing 239 
when processing edges for a current scan line 257. Edges that intersect the current scan 
line (and therefore require processing) are refenred to as active edges and are stored in 
scan-order in an ordered set of active edges 236. Edge Processing 239 generates an active 
edge &om a new edge 234 or a static edge 235 when the start co-ordinate of the new edge 
or static edge intersects the current scan line. Active edges have a sUghtly different 
format than that of new or static edges - for example, instead of having a start coordinate 
(Xs,Ys), active edges have a current_X field which gets updated from scan line to scan 
line. The current_X field corresponds to where the edge intersects (crosses) the current 
scan line. After each active edge is processed, if it is determined that the active edge will 
also intersect the following scan line, then that active edge is placed, in scan-order, into 
the ordered set of active edges for the next scan line 238. 

Fig. 23 provides more detail of the operation of Edge Processing 239 for a single 
scan line (corresponding to step 257 of Fig. 24). Before rendering the first scan line of 
the display, the ordered set of active edges 236 will be empty. This ordered set will 
remain empty until Edge Processing 239 processes a scan line which is intersected by the 
start (top) of a new edge in the ordered set of new edges 232 or a static edge in the 
ordered set of static edges 233. Until such a scan line is reached. Edge Processing 239 
has nothing to do - i.e. the ordered set of active edges is empty (determined at step 244 = 
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yes), and either there are no remaining static or new edges (step 246 = yes), or all static or 
new edges don't start until a later scan line (step 247 = no). 

If the ordered set of active edges is empty (step 244 = yes) and it is determined 
that a new edge or static edge does start somewhere on the current scan line (step 246 = 
no and step 247 = yes), then the new or static edge wiU be removed from the 
corresponding static or new ordered set and converted at step 250 into an active edge for 
processing. 

Edge Processing 239 must always process edges in scan-order (increasing X 
ordinate) along the scan line. There may aheady be active edges in the ordered set of 
active edges as a result of processing a previous scan hue (ie. step 244 = no). If there are 
already active edges, arid there are any new or static edges (step 245 = no) that start on 
the current scan Ime (step 248 = yes), then an X-ordinate of the next new/static edge must 
be compared at step 249 with an X-ordinate of the next active edge to detennine which 
should be processed next. By virtue of the fact that the three sources of edges (new, static 
and active) are stored in scan-order, the next edge for processing can always be 
determined by considering the next available edge of each source 14 (see Fig. 2(a)). 
5.4. Example of Edge Processing 

The example b Fig. 2(a) shows the desired output of a first frame of a movie 
showing a rectangle shape 14 overlapped and occluded by a triangle shape 15. A current 
scan hne 16 of display pixels has been superimposed for the purpose of this example. 
The left most display pixel of the current scan line is 13. This display pixel corresponds 
to an X-ordinate of 0. and the subsequent pixel to the right corresponds to an X-ordinate 
of 1, etc. 

Prior to Edge Processing 239 generating any output for the frame shown in 
Fig. 2(a), the Display List Compiler 608 will have generated an ordered set of new edges 
containing four new edges that describe the desired output for this frame (recall that 
horizontal edges are filtered out). These four edges are shown intersecting the current 
scan line 16 in Fig. 2(b). The edges 19 and 22 activate and de-activate the z-level 
corresponding to the rectangle, respectively. The edges 20 and 21 activate and de- 
30 activate the z-level corresponding to the triangle, respectively. 

By the time Edge Processing 239 is processing the current scan line shown in 
Fig. 2(b), the edges 19 and 22 wiU be present in the ordered set of new edges, and 
edges 20 and 21 will be present in the ordered set of active edges. Edges 20 and 21 will 
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have already been converted fiom new edges into active edges on a previous (higher) 
scan Une. 

The edges wiU be processed for the cunrent scan line as outlined in Fig. 23. In 
summary, this wiU involve, for this example, processing edge 19 first (since it has least 
X). Edge 19 will first be converted into an active edge before any crossing messages are 
generated &om it. Edges 20 and 21 will be processed next, and finally edge 22 will be 
converted into an active edge then processed. In this example, processing of the four 
edges for the current scan line generates the set of crossing messages shown in Fig. 2(c). 
The generation and foimat of these wiU be described in more detail in Section 5.8. 
5.5. Converting edges into active edges and edge persistence 

Returning now to Fig. 23, if the next edge for processing originates in the new or 
static edge ordered sets, then that edge must first be converted in step 250 into an active 
edge for the benefit of processing, otherwise the next active edge is taken at step 251. If 
the next edge for processing is a new or static edge, and that new or static edge has not 
been marked for removal, then that new or static edge wiU be placed into the ordered set 
of static edges 237 for next the frame. By this means, edges can persist over multiple 
frames. There are various ways of implementing a mechanism to allow removal of edges 
on a particular frame. One technique is to assign an identifying field to every edge, and 
for each firame, aUowing for the provision of a set of identifying fields corresponding to 
edges that should be removed on that frame. An alternative technique is to provide flags 
within a z-level referenced by edges to indicate that all referencing edges are to be 
removed. 

5.5.1 Static Edge Persistence 

An example of the operation of the static edge buffers is given in Figs. 68A-68C 
and Figs. 69A.69C. Figs. 68A-68C shows three frames 6800. 6802 and 6804 of an 
animation sequence. Fig. 68A shows the initial frame 6800. which consists of a 
house 6806. Fig. 68B shows the second frame 6802. which retains the house 6806 from 
the previous frame 6800 but adds a person 6800 standing in the doorway of the 
house 6806. Fig. 68C shows the third frame 6804, which again retains the house 6806 
but now moves the person from the doorway of the house to a position 6810 to the right 
of the house 6806. 

Figs. 69A.69C illustrate, respectively, tiie time sequence of processing operations 
that are required to produce tiie frames seen in Figs. 68A-68C. As seen in Fig. 68A, an 
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object representing the house 6806 is initially input to the Display List Compiler 608 to 
produce a New Edge Buffer 6901. which contains the new edges that are to be added to 
the current frame 6800. Fig. 16(a), described elsewhere, refers to the coordinates that 
outline the house shape, and how those coordinates are interpreted to produce the edges 
5 retained by the New Edge Buffer 6901. An Incoming Static Edge Buffer 6902, contains 
the static edges that are to be retained from the previous frame. The buffers 6901 
and 6902 are then rendered via the rendering engine 610 to output a rendered 
display 6903 of the current frame 6800. Fig. 69A also shows the Outgoing Static Edge 
Buffer 6904. which contains the static edges that are to be retained for the subsequent 
10 frame. 

In Fig. 69A, the Display List Compiler 608 processes the coordinates Fig. 16(a) 
by fransforming, stroking, morphing and sorting them. In this particular instance no 
morphing or stroking is required, but the Display List Compiler 608 still expends 
processing time in transforming the coordinates to the correct screen position and in 
sorting the edges into screen order. The processed edges are placed into the new edge 
buffer 6901, in preparation for the rendering operation. The incoming static edge 
buffer 6902 is empty, as this is the first toe 6800 in the animation (and therefore no 
edges can be retained from a previous frame). Now that the edge buffers 6901 and 6902 
are ready, the Display List Compiler 608 ceases operation for tMs frame 6800 and the 
rendering engine 610 commences operation for this frame 6800. The rendering engine 
610 merges the edges from the edge buffers 6901 and 6902 and renders them to the 
display 6903, outputting the edges that are to be retained for the subsequent frame into the 
outgoing static edge buffer 6904. Processing of the first frame 6800 of the animation is 
now complete. 

In Figs. 68B and 68C, references 6808 and 6810 respectively refer to the 
coordinates that outline the person shape that is to be rendered to the second and third 
animation frames 6802 and 6804. 

In Fig. 69B, the processing of the second frame 6802 of the animation is 
performed. Note that the incoming static edge buffer 6902 of Fig. 69B is the same as the 
outgoing static edge buffer 6904 of Fig. 69A. This is possible because the incoming and 
outgoing static edge buffers are swapped between frames in a "ping-pong" mamier. The 
outgoing static edge buffer 6904 of Fig. 69B is cleared immediately afrer the ♦'ping pong" 
swsq). 
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The Display List Compiler 608 processes the coordinates 6808 of the new edges 
that are to be added for this fiame 6802, placing the processed edges in the new edge 
buffer 6901. Hie edges of the house 6806 do not require any processing by the Display 
List Compiler 608 because they are retained ftom the previous frame in an already 
processed form. This is the reason that the static edge technique reduces the processing 
time required. 

As in the processing of the previous frame, the Display List Compiler 608 is now 
halted, and the rendering engine 610 commences its processing of this frame 6802. The 
contents of the edge buffers 6901 and 6902 are merged and rendered to the display 6903 
and the edges that are to be retained for the next frame 6804 are stored in the outgoing 
static edge buffer 6904. 

In Fig. 69C, the processing of the third frame 6804 of the animation is performed. 
The Display List Compiler 608 processes the coordmates 6810 of the new edges that are 
to be added for this frame 6804 placing the processed edges in the new edge buffer 6901. 
As in the previous frame 6802. the edges of the house 6806 do not require any processing 
by the Display List Compiler 608 because they are retained from that previous frame 
6802 in an already processed form. The edges of the person shape do require processing, 
however, as the person shape 6810 has moved and become larger when compared to the 
previous frame. Thus, the Display List Compiler 608 processes the coordinates 6810, 
placing the processed edges of the person shape in the new edge buffer 6901. 

As previously, the Display List Compiler 608 is now halted, and the rendering 
engine 610 commences its processing of this frame 6804. The contents of the edge 
buffers 6901 and 6902 are merged and rendered to the display 6903 and the edges that are 
to be retained for the next frame are stored in the outgoing static edge buffer 6904. Note 
that it is herein assumed that the house object 6806 is to be retained unaltered for 
inclusion in a fourth frame of the animation; if that were not the case then the house shape 
would not be included in flie outgoing static edge buffer 6904. 
5.6. Active Edge Processing 

Processing of a next active edge in step 252 involves generating one or more 
crossmg messages to be passed onto a subsequent module, the Z-level Activation 
Module 613. Each message includes the current-X ordinate of the active edge, along with 
the edge's left and right z-level reference. 
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If it is determined that the active edge continues down the display screen at least 
into the next scan line (in Fig. 23, step 253 = no), then that active edge is added in 
step 254 to the ordered set of active edges for processing during the next scan Une. 
5.7. Tracking Edges 
5 This process of tracking the X ordinate of an edge from scan line to scan line, as 

performed m step 252 of Fig. 23, is often referred to in prior art as "edge tracking". In 
the described system 699, edges can be straight Unes, Quadratic Bezier curves, eUiptic 
arcs or Quadratic Polynomial Fragments. The edge tracking technique for each of the 
four edge types is described below. 
10 5.7.1. Tracking Straight Lines 

Bresenham's Une algorithm is well known in the art as a method for drawing 
straight lines. Edge Processing 239 uses a modified Bresenham's algorithm to calculate 
the X-ordinate of an active straight-Une edge for each scan line m step 252. The 
C language source code in Fig. 5 shows an example of the modified Bresenham's 
algorithm being used to track (and draw) a straight-hne from a start coordinate (Xs, Y^) to 
an end coordmate (Xe, Ye). The example shows an X-ordinate of the line being calculated 
mcrementally for each Y-ordmate in the range Ys to Ye, based on three pre-calculated 
values: err, delta_errl and delta_err2. 

When data for an edge that is a straight line is generated by the Display List 
Compiler 608 (see Section 3.6.1 - Generating straight-edge trackmg parameters), it is 
generated such that it contains, at least: 

- a start coordinate (Xs, Ys), 

- an end Y-ordmate (Ye), 

- an integer gradient term (gro^, 

25 - a remainder gradient term (/ckO, 

- left flag, and 

- references to left and/or right z-levels. 

When the straight-hne edge is converted into an active straight-line edge in 
step 250, then the ient term is replaced by err, delta_errl and delta_err2 which are 
30 calculated from the data of the straight edge as follows: 
Let: 

Then: 
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err = {2xient)-ty 
delta _ errl = 2 x ient 
delta _ errl = 2 x (ient - ty) 

Then on each scan line, a new current-X ordinate for the active edge can be 
calculated according to step 252 fiom the existing cuixent-X ordinate and the grad, err, 
delta_errl and delta_err2 parameters using the modified Bresenham's algorithm as' 
5 follows: 

If the err is less than 0, then: 
err = err + delta_errl 
current _X= current_X+ grad 

Else 

10 err = err + delta_err2 

current_X = cuirentJC + grad 
If left flag is TRUE, then: 

current _X= current X- 1 

Else 

15 current_X = current_X + 1 . 

5.7.2. Tracking Quadratic Beziers 

Quadratic Beziers are described by a start, end and control point as shown in the 
example Mg. 7. When data for an edge that is a quadratic Bezier curve is processed by 
the Display list Compiler 608 (see Section 3.6.2 - Generating quadratic Bezier curve 
tracking parameters), the quadratic Bezier edge description is reformatted and presented 
to Edge Processing 239 such that it contains, at least: 

- a start coordinate (Xs, Ys), 

- an end Y-ordinate (Ye), 

- tracking parameters A, B, C and I>, 
25 - sign Flag (+1 or -1), and 

- references to left and/or right z-levels. 

For each scan line, the currentJC i^si^on of the Bezier curve is recalculated in 
step 252 of Fig. 23 based on ^. 5. C and £> as follows: 

A='A+B 

30 C=C + Z) 

Ifsign flag is +1, then: 
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Current _X = A+l^C 

Else 

Current _X = A-2-4C . 
5.7.3. Tracking Quadratic Polynomial Fragments 

The Display List Compiler 608 (see Section 3.6.2 - Generating quadratic Bezier 
curve tracking parameters) produces edge descriptions for Quadratic Polynomial 
Fragments, so that they contain, at least: 

- a start coordinate (Xs, Ys), 

- an end Y-ordinate (Ye), 

- tracking parameters^. C and A and 

- references to left and/or right z-levels. 

For each scan line, the c«rre«^_^ position of the Quadratic Polynomial Fragment 
is recalculated in step 252 hased on A, C and D as foUows: 
A = A+C 
15 c = C + D 

CurrentX = A. 
5.7.4. Tracking Elliptic Arcs 

The Display List CompUer 608 produces edge descriptions for elUptic arcs that 
contain at least: 

- equivalent circle centre (Cx, Cy), 

- equivalent circle radius 'R', 

- equivalent circle start coordinates (Sx, Sy), 

- start error value 'E', 

- circle to ellipse transformation (skew 'e' + scale T), 
2^ - end Y ordinate 'Ey', and 

- a left or right orientation. 

When step 250 of Fig. 23 converts this data into an active elliptic arc, current 
coordmates, a current error value and a current mode (described below) are initialized 



20 



30 



from this data. 



The tracking algorithm tracks coordinates relative to the center of the circle 
Hence the current coordinates are set to be the start coordinate minus the equivalent circle 
centre. TTie cmrent error value is set to be the error value provided by the Display List 
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Compiler 608. The mode is used to keep track of the octant cuirently being scanned, and 
is initialized based on the start coordinate provided to step 250. 

Note that at any stage, absolute eUipse coordinates can be derived from relative 
current coordinates (Cx, Cy) using the foUowing formula: 
5 {SE^.SE^) = C/c, + ec, + C,,c^ + C,) 

Circles are tracked using Bresenham's circle algorithm to iteratively update the 
current coordinates and error value. This algorithm, which is weU known in the art, 
tracks only the top-right octant of a circle (octant 120 of Fig. 14), and uses a sequence of 
reflections to generate the other 7 octants. Further details can be found in "Computer 

10 Graphics: Principles and Practice": Foley & Van Dam Second Edition; Section 3.3. 
Scan Converting Circles. 
5.7.5. Tracking Glyphs 

Active glyph objects maintain a current pointer into the glyph bitmap. This 
pomter is incremented row-by-row as the glyph edge is tracked. Because the left-hand 

15 edge of a glyph is vertical, the current X position does not need to be updated as the edge 
is tracked. 

5.8. Anti-aliasing and Crossing Message Generation 

Anti-aUasing is achieved by tracking edges using additional (sub-pixel) resolution. 
As is known in the art, this additional resolution can be used to determine what fraction of 

20 each pixel is occupied by a graphical primitive. This fraction (referred to m some prior 
art as pixel coverage) can be used to attenuate the contribution of the graphical primitive 
to the color of each output pixel, producing greatly improved rendering results. For 
example, the coordmates of edges can be provided with an additional 2 bits of resolution 
in both X and Y, thereby providing four times the resolution in both X and Y. 

25 A description of how Edge Processing 239 uses the additional resolution is now 

given with reference Figs. 1(a) to 1(c). Fig. 1(a) shows an active edge 0 entering a scan 
Une 1 of display pixels currently bemg rendered. In Fig. 1(b), this scan line is divided 
into four sub-pixel hnes (spel-Unes) 3. 4, 5 and 6, representing the additional sub-pixel 
resolution provided by an additional 2 bits in the Y-ordinate of the edge data. The X 

30 position of the edge is then determmed iteratively, with sub-pixel accuracy, for each of 
the four spel-lines using the appropriate one of the edge tracking techniques described 
previously. For each display pixel in which the edge being tracked intersects at least one 
spel-Une, a 4 x 4 bihnask representation of how the edge aBfects that pixel is produced. 
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Such bitmasks are often refeired to in the art as A-buffers. first described in 'The A- 
buffer, an Anti-aUased Hidden Surface Method," Carpenter, L.. Computer Graphics, 
Vol. 18, No. 3, Jul. 1984 (hereinafter "Carpenter"). A-buflfera are used to indicate which 
spels within a display pixel are afifected by an edge, and are included in the crossing 
messages passed from the Edge Processing Module 614 to the Z-level Activation 
Module 613. In this example, three A-bufifers are generated a seen Fig. 1(c), those being 
A-bufifers 8, 11 and 12, and corresponding to the current scan line pixels X = 2, X = 3 and 
X = 4. As seen in Fig. 1(c). a spel 9 is not affected by the edge, whereas a spel 10 is 
affected by the edge. 

Generation of crossing messages and their associated A-buffers is performed for 
an active edge in step 252 of Fig. 23. This step is now described in more detail with 
referoice to Fig. 3. 

In the first step 35 of Fig. 3. a current_spel_line value is initialized to 0. 
indicating that the top-most spel-line of the current scan line is to be processed first. An 
A-buffer is initialized as being blank, and a current_pixel value is initiaUzed with an X 
value corresponding to the display pixel into which the active edge enters the current scan 
line. This X value is the non-fi:actional part of the active edge's current_X ordinate. 

Next m step 36, the cmrent.X ordinate of the active edge being processed is 
compared with the current_pixel value to see if current_X is within the display pixel 
represented by current_pixel. This will be true on the first iteration as a result of step 35. 

The next step 39 operates to modify a single row of the A-buffer corresponding to 
the current_spel_line, where top-most row of the A-buffer corresponds to a 
current_spel_line = 0. The fractional data of current_X is used to determine how many 
spels should be "set" on the row of the A-buffer corresponding to the cnrrent_spel_line. 
If two bits of sub-pixel accuracy are provided by current.X, for example, then A-buffers 
can be generated that represent four spels per row as shown in the A-buffers 8, 11 and 12. 
TABLE 5 shows how two bits of fiactional data can be used to determine how to set the 
row of ttie A-buffer corresponding to the current_spel_line. 

TABLE 5 



Fractional bits 


A-buffer Row 


of cuTrent_X 


Left-most spel 


2"" spel 


3" spel 


Right-most spel 


00b 


1 


1 


1 


1 


01b 


0 


1 


1 


1 
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lOb 


0 


0 


1 


1 


lib 


0 


0 


0 


1 



Refeiring back to the example in Kg. 1(b), during processing for the top-most 
spel-line 3, (cnrrent_spel_lme = 0), the current.X value for the active edge 0 has a non- 
fractional component of 4 (100^), and a fractional component of 0.75 (fractional bits are 
lib). In this case, only the right-most spel of the top row of the A-buffer 12 is set in 
accordance with TABLE 5. 

After the A-buffer has been modified, as seen in Fig. 3 a new cnrrent_X is 
calculated in step 40 corresponding to where the active edge intersects the following spel- 
line 4. This is achieved by applying one of the tracking techniques described previously 
to the data for the active edge. The cnrreiit_spel_Ime is incremented in step 41 and if 
there are more spel-Iines to be processed, (step 42 = no), then the process returns to the 
step 36 of comparing current_X of the active edge with current_pixel. 

Jf current.X indicates that the active edge intersects the current_spel_line 
within a different display pixel to that indicated by current_pixel, then step 37 is 
performed which generates a crossing message containing, at least: 

- An X-ordinate = current_pixel 

- A-buffer 

- An activated z-level reference (copied from the left z-level reference of 
the active edge) 

- A de-activated z-level reference (copied from the right z-level reference 
of the active edge). 

Note, the z-level references can be to a null fill. 

The generated crossing message is passed as output to the Z-level Activation 
Module 613 via a sorting means (e.g., a sort buffer). The sorting means is necessary to 
ensure lhat the Z-level Activation Module 613 receives crossing messages in scan-order 
(i.e., sorted by increasing X-ordinate). 

The A-buffer is then re-initialized to be blank, and the current _pixel is set to be 
the non-firactional part of the current.X value for the active edge (0). The A-buffer is 
then modified to be the A-buffer 11 as previously described to indicate the effect of the 
active edge on the current_spel_line 5 of the current_pixel. 
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Processing continues for the remaining spel-line 6, which causes the crossing 
message containing A-bufifer 11 to be ou^ut. 

Once all spel-lines have been processed (step 42 = yes), then a final crossing 
message containing A-bufifer 8 is generated and output at step 43 as described previously. 
5.8.1. Generating Crossing Messages for Glyphs 

Each row of a rendered glyph is spUt into byte-wide (or 8 pixel wide) blocks that 
are processed in turn. Each byte is used as an index into a 256-entry look-up table (LUT), 
with each entry occupying 32 bits. A single entry in the LUT identifies at which pixels 
an 8 pixel row crossing messages must be generated. 

Each entry is processed in nibbles - the first nibble records the number of 
crossings generated by the byte, and each subsequent nibble recoixis the position of a 
crossing relative to the begimiing of the region. A special entry recording eight crossings 
but no positions is used for the unique case of a byte with a crossing at every pixel. 

Referring to Kg. 26, an example with a bitmap 265 of width 16 and height 2 is 
shown. The binary representation 265 would be {0xD6, 0x54, 0x99, OxCC} (note that bit 
'7' within each byte refers to the leftmost pixel in the corresponding pixel block). Taking 
Ihe first byte of this bitmap, and looking up its unsigned value in a LUT 266, it is seen 
that there are 6 crossings at positions 0, 2, 3, 4, 5 and 7. Assuming that the LUT 265 is 
placed at screen coordinates (100, 100), this would mean that crossing messages must be 
generated for (100, 100), (102, 100), (103, 100), (104, 100). (105, 100), (107. 100). A 
byte that generates 8 crossing messages can also be observed in this figure. 

After the LUT 266 has been indexed and a set of crossings for the byte obtained, 
an extra crossing may be inserted or an incorrect crossing deleted. This is governed by a 
flag that is initiaUzed to 0 at the beginning of each row of the bitmap. When this bit is 
cleared, crossings are umnodified. When the bit is set, if a crossing corresponding to the 
left-most position in the pixel block (a "zero crossing") does not exist, one is inserted. 
Conversely, if a zero crossing exists, it is deleted. 

The status of the flag is updated at the end of processing of each byte in the row. 
An odd number of crossmg messages inverts the flag, whereas an even number of 
crossing messages maintains the current value of the flag. If the flag is set at the end of a 
row, then a final crossing message is output to terminate the row correctly. The position 
of this message is at the start of the pixel block immediately to the right of the current 
pixel block. 
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Referring back to Fig. 26, it is possible to trace the status of this flag across the 
four bytes in the bitmap. At the beginning of the first row, the flag is initialized to 0. The 
first byte resolves to an even number of crossings, so the flag is not toggled at the end of 
this byte. The second byte, however, resolves to an odd number of crossings. As a result, 
the flag is toggled to 1. The second byte is the last in the row, and hence a final crossing 
message is generated to finish the row. 

Moving now to the second row, the flag is initialized to 0. The third byte resolves 
to an odd number of crossing messages, resulting in the flag being set to 1 at the end of 
this byte. The fourth byte resolves to 8 crossing messages, at 0, 1, 2, 3, 4, 5, 6. and 7. 
However, the flag is set. As a zero crossing message exists, this message is deleted, 
resulting in 7 crossing messages being output. As this is an odd number of crossings, the 
flag is again toggled to 0. As processing is now at the end of the row and the flag is not 
set, no final crossing message is output. 

The crossing messages in aU cases are provided with S-bu£fers that have all sub- 
pixels set to -1. As a result, glyphs should always be rendered using the odd-even 
winding rule. 

5.8.1.1. Generating Glyph Crossing Messages and anti-aliasing 

Fig. 66 shows the same scan lines as shown in Fig. 26. Considering a single 
pixel 6601, on that scan line, a crossing message 6602 is generated. The crossing 
message 6602 has all sub-pixels set to -1. This crossing message will result in a whole 
pixel 6603 on output. In effect, glyphs are not anti-aliased and sub-pixels are not 
considered. This is true for aU crossing messages generated for a glyph, including those 
added in consideration of a "zero-crossing" flag. 
5.9. Example of Crossing Messages 

Continuing the example shown in Fig. 2(c), shows crossing messages produced by 
the Edge Processing Module 614 for the active edges as shown in Fig. 2(b). In this 
example, 4x4 anti-aliasing is used, where active edge coordinates are described with an 
additional 2-bits of resolution corresponding to sub-pixels (spels) within display pixels. 
Processing of the edge 19 will be performed first and therefore a corresponding crossing 
message 23 will be generated first. Since the active edge will have had a left Z-level 
reference describing the fill for the rectangle, then the activated Z-level reference of the 
generated crossing message 23 wUl have an activated z-level reference describing the fill 
for the rectangle. 
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Processing of the active edge 20 occurs next As the module 614 (outlined in 
Fig. 3) iterates through the first three of the four spel-lines for active edge 20. the crossing 
message 25 is generated. Because the active edge intersects the fourth (bottom-most spel- 
line) on a different pixel, a different crossing message needs to be generated for this, and 
so crossing message 24 is generated next. Both these crossing messages have activated z- 
level references describing the fiU for the triangle shape. 

Similarly, the crossing messages 26 and 27 are generated for active edge 21, and 
these have deactivated Z-level references to the fill for the triangle shape (copied from the 
right z-level reference of the original active edge 21). Crossing message 28 is generated 
last from the edge 22, and has a deactivated z-level reference to the fill for the rectangle 
shape. 

5.9.1. Another Example 

Fig. 70 shows a fiorther example that illustrates the updating of S-buflfers with 
receipt of zero-crossing messages. Mtially, for a pixel #1 in a scan line, there are no 
edges crossing and hence there is no crossing message or S-bufifer„ for that pixel. As 
such, for that pixel #1. an S-buffer„ is set populated with zeros. The most right-hand 
column of this S-buffer„ is then copied to the first (ie. left-most) column of an S-buffern-i 
for the next pixel #2. That column, is then copied across the columns of that S-buffer„.i 
which, m this case, results in that buffer also being populated with zeros. As illustrated, 
an edge crosses pixel #2 and, according to the illustrated rules for crossings of sub-lines 
(for crossing 4, 2 or 1 sub-hne), a zero-crossing message for pixel #2 is detennined. As 
seen, that message is populated with +rs in its lower right-hand quarter, thus 
complementing the edge crossing the scan line. For pixel #2, the zero-crossing message 
and the S-buffer„.i are summed to give the S-buffer„ for that pixel. In this case, such is 
merely the crossing message since the buffer was null. For the next pixel #3, again the 
right-most column is copied from the S-buffer„ and populated in the S-buffer„.i. in this 
case by ones. The crossing message at pixel#3 is detemiined according to the crossing 
rules where it is seen the relevant edge crossed only 2 sub-Unes in the top-right quadrant 
of the pixel. Again, the crossing message and the S-buffer„.i are summed to give the S- 
buffer„ for pixel #3. For pixel #4. again the right-most column is copied and populated 
across the S-buffer„.i. A crossing message is again determined using the rules and 
summed to give the S-buffer„ for pixel #4. This described approach to using crossing 
messages and S-buffers to effect antiaHasing affords the ability to accurately determine an 
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active level for a pixel location on a scan line without a need to composite levels on a 
sub-pixel basis. AU sub-pixel processes occurs using S-bu£fers and crossing indicators, 
thus leaving compositing duties at the pixel level. 
5.10. Re-ordering of Active Edges and Crossing Messages 

Although active edges for a scan line are processed in scan-order, the 
corresponding crossing messages may not necessarily be generated and output in scan- 
order (increasing X). For example, when the edge 20 is processed in Fig. 2(b). a crossing 
message 25 is generated and output first, followed by message 24. Crossing message 25 
has a higher X-ordinate than the message 24 and so the crossing messages are not output 
in scan-order. Further, the result of calculating the new current.X during operation of 
the Edge Processing Module 614 may cause a first active edge to have a lower scan 
position than a second active edge akeady processed on this scan line. 

An example of this situation is given in Kg. 21. There, the two dashed horizontal 
lines 230 and 231 represent scan lines of the output display. The upper dashed line 230 
represents a scan line currently being processed by the Edge Processing Module 614, and 
the lower dashed line 231 represents a next scan line to be processed. Kg. 21 shows ihree 
active edges 221. 222 and 223 that intersect the cmxent scan hue 230 at crossings 224, 
225 and 226 respectively. The current.X field of the active edges indicates the position 
of the intersection. The Edge Processing Module 614 calculates a new current.X value 
for each of the active edges coixesponding to the intersections 227. 228, 229 on the next 
scan line 231. In this example, these modified active edges are no longer in scan-order 
because active edge 223 has crossed active edge 221 and active edge 222. The modified 
active edges therefore require re-sorting before they are placed in the ordered set of active 
edges for the next scan line. In the example, the desired order of active edges in the 
ordered set of active edges for the next scan line is first 223, then 221. then finally 222. 

The Edge Processing Module 614 inserts the modified active edges into a sorting 
means ie.g., a sort buffer) such that they are in scan-order before they are transferred into 
the ordered set of active edges for the next scan line. It is also necessary to pass outgoing 
crossing messages (as generated in step 252 of Kg. 23) through a sorting means (e.g. a 
sort buffer) so that the Z-level Activation Module 613 receives crossing messages in scan- 
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6. THE Z-LEVEL ACTIVATION MODULE 

The Z-Ievel Activation Module 613 uses the edge intersection data passed to it 
from the Edge Processing Module 614 to keep track of which z-levels contribute to the 
output color as the frame is rendered. A stream of output pixels is to be generated by the 
rendering engine 610 in scan order. Each crossing message from the Edge Processing 
Module 614 represents a display coordinate for which the required output color changes 
when produced in scan-order. In the foUowing description, one or more scan-order 
contiguous pixels starting at a specified raster space location are referred to as a run of 
pixels. The description assumes ttiat A-bufifers are generated for edge crossings, that is, 
for the case when the rendering system 610 is performing anti-aUasing. 

6.1 . The Ordered Set of Interesting Z-levels 

The Z-level Activation Module 613 maintains a list of "interesting" z-levels, seen 
generally in Fig. 59. By mteresting, it is meant that the z-level in question miist be 
considered by the Z-level Activation Module 613. The Z-level Activation Module 613 
uses the ordered set of interesting z-levels to convert the input of crossing messages into 
an output of region messages indicating which z-levels contribute to tiie output color for 
runs of pixels along a current scan line. That is, the runs lie within tiie points described 
by two adjacent arossiog messages. 

The operation of the Z-level Activation Module 613 can also be seen in terms of 
converting a stream of crossing messages to an output of region messages which indicate 
which depths are present on the span and what their relative depth ordering is over tiie 
span. Span in this regard means tiie intemal on tiie scan line bounded by crossing 
messages. Since z-levels bind depths to fills, such an ordered set of deptiis can be used to 
determine which z-levels contiibute to tiie output color for runs of pixels along a current 
scan line. This is ttie role of tiie compositor, to be considered later. Conversely, 
determining tiie set of z-levels present on a span is equivalent to determining tiie deptiii 
present on a span. 

6.2. The Flow of Control in the Z-level Activation Module 

As wifli tiie Edge Processing Module 614. flie Z-level Activation Module 613 
considers flie task of rendering on a scan hne by scan line basis, and expects to receive 
input (crossing messages) in scan-order. The fimctionaUty of tiie Z-Level Activation 
Module 613 for a single scan line is summarized by Fig. 58. Processing for a scan line 
begins by initializing tiie list of active z-levels to be empty, and setting a Current_X 
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value to 0 in step 628. Current_X is used to indicate the pixel currently being processed, 
and a value of 0 represents the first Oeft-most) pixel of the scan line. 

Crossing messages for the scan line are processed in left-to-iight order 
(increasing X). 

If there are no edge crossing messages for this scan line, as determined by "no" at 
step 627, then immediately the Z-level Activation Module 613 will request rendering of 
the entire scan line at step 629. Because at this stage the Ust of interesting z-levels is 
empty, this will result in the scan line being rendered with a default background color 
only. 

If crossing messages for the current scan line do exist (step 627 = yes), then the X 
ordinate of the next available crossing message is compared with the Current_X value at 
step 630. If Current_X and the X ordinate of the crossing message are not equal, then 
the Z-Level Activation Module 613 must request the rendering of the run of pixels 
between Current.X and the X ordinate of the crossing message before processing this 
crossing message. This request causes, at step 631, a Compositing Module 612 in the 
rendering engine 610 to generate output colors for this region of pixels, using the contents 
of the ordered set of interesting z-levels. 

In the case that Current_X and the X ordinate of the crossing message are equal 
("yes" in step 630), then the crossing message corresponds to color generation for the 
current pixel, and so processing of the crossing message proceeds. 
6.3. Activating and Deactivating Z-levels: Winding Counts 

To facilitate rendering of self-intersecting objects, the Z-level Activation 
module 613 utilizes winding-rules. Rather than storing just a Boolean indication of 
whether or not a z-level has been activated, the module stores a small winding count 
integer value per active z-level. Winding counts are well known to one skilled in the art. 
Two of the winding rules that can be used are commonly described in prior art: non-zero 
and odd-even (parity). A third rule Oess common, but still described in the prior art) that 
can be used is negative winding, described in Section 6.7 - Converting S-buffers to A- 
bulTers; Winding Rules, below. 

Crossing messages may contain a reference to a z-level that is to be 'activated', 
te.. the z-level must have its winding count decremented. Crossing messages may also 
contain a reference to a z-level that is to be 'de-activated', i.e., the z-level must have its 
winding count incremented. Hiis incrementing and/or decrementing of the z-levels only 
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appHes to the portion of the pixel indicated in the A-Bufifer of the crossing message, as 
shown in Fig. 2(d). A-buffers 29, 30 and 31 are examples of the sub-pixels that are 
activated by the crossing messages 23, 24 and 25. 32, 33 and 34 are examples of the sub- 
pixels that are deactivated by the crossing messages 26, 27 and 28. 

The effect of these activations and deactivations to the contribution of a z-level to 
the output pixels depends on which winding rule is used with that z-level. 

6.4. The Ordered Set of Interesting Z-Levels - continued 
A 2-dimensionaI array of winding counts is stored by the Z-level Activation 

Module 613 for each z-level in the Hst of interesting z-levels, as seen in Fig. 59. A 2- 
dimensional array of winding counts is termed an 'S-buffer'. Each winding count 
represents the state of the z-level for one of the sub-pixels within the current pixel being 
rendered (643, 647 and 651). This allows the Z-level Activation Module 613 to keep 
track of which portion of the current pixel each z-level is active over. The order of the 
Ordered Set of Interesting Z-levels is depth order. In the example shown in Fig. 59, z- 
15 levels 640, 644 and 648 are ordered by their depths; 641, 645 and 649. The fill data 642, 
646 and 650 define the color of the z-levels 640, 644 and 648. 

6.5. Adding new Zr-levels to the Ordered Set of Interesting Z-levels 

Returning to Fig. 58, if a new crossing message does not have an activate z-level 
(step 638 = no), then processing steps related to the activate z-level are not performed. 

20 Similarly, if a crossing message does not have a deactivate z-level (step 639 = no), then 
processing steps related to the deactivate z-Ievel are not performed. 

If a new crossing message activates a z-level, and that z-level is not already in the 
Ust of interesting z-levels (step 632 = no), then in step 640a the Ust of interesting Z-levels 
is checked. If fiall, a Z-level with all zero winding counts is removed from the Ust. After 

25 this, a new entey, corresponding to that detected in step 638, is inserted into this Ust such 
that the Ust maintains z-level depth order, as shown in step 634. The A-buffer of the 
crossing message is used to initiaUze an array of sub-pixel counts {j.e., the z-level's S- 
bufifer) that is to be associated with the newly inserted z-level. The sub-pixels mdicated 
by the A-buffer correspond to sub-pixel counts in the S-bu£fer that should be initialized 

30 with a value of minus one. The remaining sub-pixel counts are initiaUzed with a value 
of 0. Examples of this conversion &om an A-Bufifer to an array of winding counts are 
shown in Fig. 2(d). 
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If a new crossing message activates a z-level that is already in the Onlered Set of 
Interesting Z-levels (632 = yes), then the A-bu£fer of that crossing message is used to 
determine which sub-pixel counts in the S-buffer associated with that z-level are to be 
decremented. 

still with reference to Fig. 58. if a new crossing message deactivates a z-level that 
IS not aheady in the Ust of interesting z-levels, then a corresponding new entry is inserted 
at step 636. into this Hst such that the Ust maintains z-level depth order. The A-bu£fer of 
the crossing message is used to initialize an array of sub-pixel counts ii.e., the z-level's S- 
buffer) that is to be associated with tiie newly inserted z-level. The sub-pixels indicated 
by tiie A-buffer correspond to sub-pixel counts in the S-bufFer that should be mitialized to 
plus one. The remaining sub-pixel comits are mitialized with a value of 0. Examples of 
this conversion from an A-Bufifer to an array of winding counts. S-buffers. are shown in 
Fig. 2(d), where the crossing messages with X = 14. X = 15, and X = 15 are crossing 
messages that deactivate a z-level. 

If a new crossmg message deactivates a z-level that is already in the list of 
interesting z-levels (635 = yes), then tiie A-buffer of that crossing message is used to 
determine which of the sub-pixel counts m the S-buflfer of tiiat z-level are to be 
incremented. For each sub-pixel mdicated in tiie A-buffer, tiie winding count of tiie 
corresponding sub-pixel in tiie S-Buffer is incremented at step 637. 

Kg. 71 illustrates an alternate approach to handling crossing messages to tiiat 
shown in Fig. 58. m Fig. 71, all reference numerals tiiat correspond witii tiiose shown m 
Fig. 58 have tiie same function. Fig. 71 differs from Fig. 58 in tiiat step 640a of Fig. 58 
has been removed, such tiiat step 634 follows directly from a "no" indication of step 632. 
Fig. 71 also includes an additional step 641a which operates following step 637 to check 
if an entry in tiie Ust of interesting Z-levels has winding counts that are all zero. If so, tiiat 
entry will no longer be effective and tiius is removed from tiie Ust. As such tiie metiiod of 
Fig. 71 operates to remove entiies from tiie Ust when tiiey can no longer contribute to flie 
unage being rendered, and contirasts Fig. 58 where entries are removed only when space 
is required in tiie list for new entries. 

6.5.1. Maintaining an Ordered Set oflnteresting Z-levels in Hardware 

Discussion can now tiim to that of a hardware implementation, such as an ASIC 
The Z-level Activation Module 613 manages Z-levels, on a scan line basis, in a Z-level 
Activation Table (ZLAT). as shown in Fig. 52(b). As a scan Une is processed from left to 
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right, Z-levels referenced by crossing messages are graduaUy introduced to the ZLAT. 
Each incoming Z-level is assigned a ZLAT entry to store an S-bufifer 573, fiU 
infonnation 572, and is addressable by Z-level depth 571. 

Z-level entries in the ZLAT are not ordered by depth. Rather, they are most likely 
to be arranged in the order they were received in crossing messages. Fig. 52(a) shows an 
example of 5 Z-levels that are active at a particular X-ordinate, and their relative depths, 
while Fig. 52(b) illustrates that the respective 5 Z-level entries in the ZLAT are not 
ordered by depth. Z-level entries are maintained in the Z-level Activation Module 613 in 
order of decreasing depth by means of a map 570. The map 570 is a table containing the 
same number of entries as the ZLAT, and is an ordered Ust of local depths that each 
include a mapping reference to respective entries in the ZLAT. For example, Z-level 555 
has a local depth of 1 (it is the second highest Z-level, where a local depth of 0 is the 
highest) and is mapped to ZLAT entry 3; that is, Z-level 555. Also illustrated is an 
updated map 574. 

Using this mechanism, Z-level entries can be accessed in two ways. The ZLAT of 
the present disclosure is implemented as a content addressable memory, such that 
supplying a Z-level d^th will return a corresponding Z-level entry. This enables the size 
of the ZLAT to be varied according to the number of objects active in a span whilst 
permitting ready inclusion and removal of Z-levels from the ZLAT. The ZLAT is 
accessed in this manner when updating an S-buffer's state as crossing messages are 
received. Conversely, ZLAT entries are also addressable by local depth via a map table. 
That is, the Z-level present at a particular local depth can be determined by applying the 
m^ to the local depth. The ZLAT is accessed in this manner when determining the 
ordered subset of depths that are active on a particular span for compositing. 
6.5.1.1. Adding a new Z-level Entry in Hardware 

Z-levels are added to the ZLAT the first time they are referenced by a crossing 
message on a scan line. On subsequent references, the respective entry only requires 
updating, and it remains in the same location within the ZLAT. An input counter is 
maintained to track the location in the ZLAT that the next new Z-level entry will be 
inserted in to. The counter is initially reset to position 0 at the beginning of a scan line 
and is incremented as each new Z-level is added to the ZLAT. Therefore, up until the 
ZLAT becomes full, Z-level entries will be maintained in the order they were received in 
CTOssing messages. 
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At the begmning of a scan line, the ZLAT is initiaUy empty. The map 570 is also 
in an initial state such that local depfli 0 contains a m^ing to ZLAT entry 0, and local 
depth 1 contains a mapping to ZLAT entry 1. and so on. When a new Z-level is then 
added as an entry to the ZLAT. it is inserted in the position pointed to by the input 
comiter. the map 570 is updated to reflect any changed ordering of Z-levels. and the input 
counter is incremented. To update the map. the local depth of the incoming Z-level must 
be determined relative to the local depths of the Z-levels currently being maintained in the 
ZLAT. That is. determining which Z-levels have a local depth above the incoming Z- 
level and those having a local depth below the incoming Z-level. 

This process is illustrated in Figs. 53(a) and 53(b). As a first step in this process, 
seen in Fig. 53(a), Z-level entries currently being maintained in the ZLAT 578 ari 
compared with the Z-level to be added. ZLAT entries are marked as being locaUy above 
or below the incoming Z-level by comparing thdr Z-level depths. In Fig. 53(a). the 
ZLAT entries are marked as being above (mark = 0) or below (mark = 1) the Z-level 575 
1 5 being added in the list 577. 

The second step is to reorder markings in the order of thek associated local depths 
as determined by the map 579. This has the effect of discriminating between local depths 
in the m^ that are above or below the local depth of the incoming Z-level. Using the 
reordered markings 576. the map 579 is updated to reflect the new ordering of local 
20 depths due to the addition of the new Z-level. 

To update the map 579. local depths that are marked as being below that of the Z- 
level to be added, seen at 575, are demoted in order, while the new local depth is 
promoted to produce the next map 580. Note that the local depths of respective ZLAT 
entries change, however, as ZLAT entries are maintained statically within the ZLAT, 
their mappings remain unchanged. For example, in the ZLAT 578. Z-level of d^th 9 hai 
a local deptti of 4 in the current map 579 before the new Z-level 575 is added. Its 
mapping is to position 1 in the ZLAT 578. and this mapping remains michanged even 
after its local depth is demoted to 5 in the next map 580 after the new Z-level 575 is 



25 



30 



added. 



Kg. 53(b) illustrates another example of adding a further Z-level 586 to the 
ZLAT 581 arising ftom Fig. 53(a) and the steps for updating the map 584. 585 
^propriately. 

6.5.1.2. Re-sorting a Z-levei Entry in Hardware 
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As an extension, Z-level entries can be maintained in the ZLAM in order of 
interest followed by decreasing Z-level depth. Interesting Z-levels are those for which 
appUcation of their Fill Rule on their S-buflfer results in a non-zero A-buffer. That is. an 
interesting Z-level is a potentially contributing one. While it is not possible for a' Z- 
level's depth to change, there are circumstances where a Z-level's state changes from 
interesting to not interesting, and vice versa. This occurs in two possible scenarios. It can 
occur when the Z-level Activation Module 613 receives a crossing message that 
references a Z-level that aheady resides within the ZLAT. and die A-buffer also 
associated with the crossing message, causes the Z-level to change its state of 
interestingness. It can also occur at the completion of processing a span in which the 
right-most spels of S-buffers of ZLAT entries must be copied across the entire S-buffer. 
This step can have the effect of making an interesting Z-Ievel unmteresting. 

The event of a Z-level changing its state of interest has no effect on entries m the 
ZLAT. except the updating of the S-buffer of the respective Z-level. It does, however, 
require the map to be updated to reflect the new local depth ordering. Thus, local-depths.' 
maintained by the map. need to be resorted rattier than insertion-sorted. 

Fig. 54(a) illustrates a scenario where a Z-level 587 residing in the ZLAT 590 
becomes uninteresting. The current state 599 of the ZLAT 590 shows where some depths 
are interesting and some are not. and the cmrent map 591 reflects the current local depth 
ordOTng. This can be used to form a partitioning of the ZLAT 590. To update the 
map 591, marks are made against entries in the ZLAT 590 in a similar manner as to the 
process of adding a new Z-level. Entries are marked as being above or below die Z-level 
changing state, with respect to the local depth ordering. In the marks column 589, entries 
are marked with a 1 if their local depth ordering wiU be above the changing Z-level once 
it has been updated. The markings are also sunilarly reordered to reflect local depth 
statuses 588, and the map 591 is iq,dated to the map 592 by promoting local deptii 
mappings that are above the changing Z-level. and demoting the mapping of die changing 
Z-level to a lower local depth. 

Fig. 54(b) Ulustiates this process when an enty 593 is becoming interesting 600 
Jn this example, entries in the ZLAT 596 are marked with a 1. as shown in the marks 
column 595. if flieir local depth will be below that of changing Z-level 593. Marks are 
reordered 594 simUarly, and the map 597 is updated to 598 so tiiat marked local depth 
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mappings are demoted to lower local depths, and the changmg Z-level's local depth is 
promoted. 

6.5.1 .3. Replacement of a Z-Ievel Entry 

A mechanism is not required for removing inactive ZLAT entries on the current 
5 scan line. A lazy poKcy can be implemented whereby inactive entries are ignored unless 
a new Z-level is to be added and there is no longer room in the ZLAT, given its finite 
size. In such situations, the Z-level entry mapped to by the lowest local depth is removed 
and the Z-level being added takes its entry in the ZLAT. The map table is updated as if 
the entry in the ZLAT being claimed was empty. 

10 6.5.1.4. Simplified ZLAM Management on a Single Scan line 

Figs. 65 A and 65B provides an overview of how the ordered set of interesting z- 
levels changes over a scan line, as it is updated by flie method described in Fig, 58. For 
clarity, optional lazy poUcy of inactive z-level reclamation is described in Section 6.5.1.3 
is not shown - inactive z-levels are removed firom the table. 

1 5 Fig. 65A represents an image 6500 formed of a rectangle 6502, a circle 6504 and 

a triangle 6506 being rendered on a particular scan line 6508 having x-ordinates on the 
range [0,18]. Fig. 65B shows, with respect to edges active on the scan line 6508, an 
ordered set of interesting levels 6530 forming the z-level activation table upon 
encountering each of the edges. 

20 At 6511, the left hand edge associated with the rectangle 6502 intersects pixel #4 

on the scan line 6508 and a crossing message is generated. This crossing message leads 
to the z-level 6521 associated with the rectangle 6502 being added to the ordered 
set 6530. 

At 6512, the left hand edge associated with the triangle 6506 intersects pixel #6 
25 and a crossing message is generated. This crossing message leads to the z-level 6522 
associated with the triangle 6506 being added to the ordered set 6530 of interesting z- 
levels. In this case it is seen that the appUcation of the method of Fig. 58 results in the fill 
for the triangle 6504 being inserted topmost in the ordered set 6530. Fills are inserted 
such that the set 6530 is maintained in depth order, and the triangle 6506 lies able the 
30 rectangle 6502, 

Sinularly at 6513, the z-level or fill 6523 for the circle 6504 is inserted between 
the z-levels 6522 and 6521 of the triangle 6506 and rectangle 6502, respectively. At 
6514, the removal of the fill 6521 of the rectangle 6502 as such becomes inactive does not 
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effect the ordering of the ordered set 6530. At 6515. the circle 6504 becomes inactive 
and its fiU 6521 is removed from the ordered set 6530 which then becomes empty. 
6.6. Processing Runs 

This description now returns from the discussion of issues more pertinent to 
hardware, and continues the more general method description from Section 6.5. 

Processing of crossing messages continues until there are no further crossing 
messages corresponding to Current_X (630). When this happens, the Z-level Activation 
Module 613 must request the Compositing Module 612 to generate output color 631 for 
the run of pixels before proceeding to process any further crossing messages, as 
illustrated in Fig. 40. 

Fig. 40 illustrates outputting run data to the Compositing Module 612 and the 
associated tracking of runs of pixels where the set of interesting z-levels does not change. 

Firstly, generation of the color for the display pixel Current_X must be requested 
at step 399. At this time, the contents of the ordered set of interesting z-levels describes 
all the z-levels which contribute to the color of this pixel, and sub-pixel coverage 
infonnation is provided for each z-level by its S-buffer. The remainder of Fig. 40 wiU be 
described in Section 6.8. 

6.7. Converting S-bujEfers to A-buffers: Winding Rules 

The Compositing Module 612 takes A-buffers associated with each interesting z- 
level as input. Therefore the Z-level Activation Module 613 converts the S-buffer 
associated with each interesting Z-level into an A-buffer before passing run data to the 
Compositing Module 612. hi domg so, the Z-level Activation Module 613 utilizes a 
windmg rule. The three winding rules mentioned above are illustrated in Fig. 57(a) and 
Fig. 57(b) where S-buffers 624 and 620 are converted to A-buffere 621, 622, 623 
and 626, 625, 619 by each winding rule, respectively: 

■ Odd-even: the fill contributes to the sub-pixel when winding counts are 

odd. as shown in the A-buffers 623 and 619; 

- Non-zero: the fiU contributes to the sub-pixel when winding counts are 
other than zero, as shown in the A-buffers 621 and 626; and 

- Negative winding: the fill contributes to the sub-pixel when winding 
counts are negative, as shown in the A-buffers 622 and 625. 

The results of these winding rules over a whole shape 421 are illustrated in 
Kg. 43(a). In Fig. 43(a) a number of winding counts 420. 426 and 422 over the scan 
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line 424 are shown. Similarly, winding counts 429, 427 and 423 exist over a scan 
line 425. In Fig. 43(b) the results of applying Non-zero winding 428, Negative 
winding 430 and Odd-even 431 rules, given that the edges of the shape define the hatched 
fill, seen in the representations of Fig. 43(b). within the bounds of the shape. 

The negative winding rule is particularly usefiil for producing and rendering 
stroked edges using a minimal amount of calculation. The process of stroking a path has 
been described herein as part of the Transform, Morph and Stroking Module 616 (see 
Section 3.4). An example of part of a stroked path is shown in Figs. 36(a) to 36(c). 
Fig. 36(a) shows an ordered set of three edges 362. 361 and 360 that are to be stroked and 
which form a clockwise triangular path firom an origm 359. The ordered set of edges to 
be stroked are provided along with a 'null' left z-level 366, a ri^t z-level 367 and a 
stroke z-level (i.e.. a pen color) 365. An example of the desired output for the sh^e 
represented by the ordered set of three edges and associated z-levels is given in 
Fig. 36(b). To achieve edges that describe the desired output shown in Fig. 36(b), the 
15 Transform Morph and Stroking Module 616 will have produced left-hand stroke 
edges 355 and right-hand stroked edges 356, drawn in Fig. 36(c), which shows a large 
scale representation of the top-most join at the origin 354. hi the case where the stroke z- 
level 365 does not have transparency, then the stroking process will have assigned: 

- the null left z-level 366 to the left z-level reference of all left-hand stroke 
20 edges 355; 

- the stroke z-level 365 to the right z-level reference of all left-hand stroke 
edges 355; 

- the stroke z-level 365 to the left z-level reference of aU right-hand stroke 
edges 356; and 

- right z-level 367 to the right z-level reference of all right-hand stroke 
edges 356. 

Kg. 36(c) depicts two scan Imes 357 and 358. Above each of the two scan Unes 
are numbers representing the winding count of the right z-level reference 367 of the 
origmal set of edges at diflferent points along the scan line. If either odd-even or non-zero 
30 windmg rules are used, then as can be seen m Fig. 36(c), an undesired triangular area 368 
formed by self-intersecting edges is enclosed by the right-hand stroke edges would be 
rendered with the z-level 367 active since the winding count in this region is T. By 
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using the negative winding rule, a winding count of «1' is considered inactive and 
therefore the join wiU be rendered coirecfly. 
6.8. Processing Runs, Continued 

Returning to the discussion of Fig. 40 - Processing Runs, using the A-buffers and 
the ordered set of interesting fills, the Con^ositing Module 612 in step 399 calculates an 
anti-aliased color for ttie single pixel. 

The list of interesting z-levels is also used to generate color for subsequent pixels 
along the scan line up to the X ordinate of the next crossing message. However, the array 
of winding counts (the S-bu£fer) for each z-level must first be modified to indicate the 
correct coverage of each z-level for the subsequent pixels, since it may not be the same as 
for pixel Current.X. For these subsequent pixels, the coverage of each z-level does not 
change until the X ordinate of the next crossing message. 

Referring again to the example Fig. 2, the left edge of the rectangle 14 caused the 
Edge Processing Module 614 to generate the crossing message 23 pc = 4). The A-buffer 
indicates that for the pixel X = 4. the z-level used to fill the rectangle is active for the four 
sub-pixels in the bottom-iight comer of the output pixel. However, to correctly render the 
rectangle, this z-level should also remain active for the lower two sub-scan lines on all 
subsequent pixels along the scan line until the right hand edge at X = 15. where the z- 
level is deactivated. A correct S-buffer representing the contribution of the z-level to 
display pixels X = 5 up to but not including X = 15 can be generated by copying the right- 
most sub-pixel value to all other sub-pixel values on the same row, as shown in Fig. 8. 
Fig. 8 shows an S-buffer 65 corresponding to the crossing message 64 for the pixel X = 4 
in Fig. 2. An S-buffer 66 represents the result of copying across the right-most winding 
counts to all other winding counts for each row. This result correctly represents the 
winding counts for the z-level used for rendering the rectangle for pixels X = 5 up to 
X = 14 inclusive. 

Referring back to Fig. 40. step 400 operates to copy across the right-most winding 
counts for all z-levels in the ordered set of interesting z-levels, and Current_X is 
incremented. The Compositmg Module 612 can now be invoked for the remaining pixels 
in the run. according to step 401. If no crossing messages remain for the current scan 
Une, the Compositing Module 612 is requested to generate pixels from Current.X to the 
end of the current scan line. 
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Note that the color generated for these pixels may still vary across the run, such as 
in the case where one or more of the active z-levels is a blend or bitmq>. It is oily the 
coverage information and the z-levels involved that remain constant over the run. The 
details of pixel color generation based on the contents of the list of interesting z-levels are 
5 in the description of the Compositing Module 612 (see Section 7). The last step 402 of 
Fig. 40 is to modify Current.X to be that of the X ordinate of the next available crossing 
if there is one available. 

The procedure for generating an A-buffer from the S-buffer of each of the z-levels 
in the ordered set of interesting z-levels is the same as that for compositing single pixels 
10 and has been described in section Converting S-buffers to A-buffers; Winding Rules, 
above. 

Finally, to return to Fig. 58, edge crossing messages are processed for the current 
scan line until none remain, this being determined at step 627. The operation of Fig. 58 
repeats for all scan lines to be rendered. 
15 7. COMPOSITING MODULE 

The compositing module 612 takes as its input each of the foUowing: 

- the ordered set of z-levels and their respective A-buflFers as produced by 
the Z-level Activation module 613, 

- coordinates in the output display space defining the length of the run of 
20 output pixels to be produced, 

and produces as its output: 

- a run of pixels suitable for presentation on the output display device. 
The ordered set of input z-levels is sorted in depth order. The concept of z-level 

depths has been previously described. In the foUowing discussion, the terms top and 
bottom are used. Given two z-levels, A and B, if the final ou^ut portrays z-level A as 
obscuring z-level B, then z-level A is considered to be on top of, or above z-level B. 
Conversely, z-level B is considered to be below or beneath z-level A. In addition, the 
topmost z-level is the one that is considered to be conceptually 'on top' of all the others, 
in that it may obscure aU of the others, and in turn be obscured by none of the others. 
30 Conversely, the bottom-most z-level is the one that can obscure no other z-levels, but can 
be obscured by aU other z-levels. In addition, there is always present a background z- 
level, which is considered to be below the bottom-most z-level in the input set. The 
background z-level is not included in the ordered set of input z-levels, and any discussion 



25 



wo 2004/114223 



-74- 



PCT/AU2004/000842 



Of z-levels that are part of the input set does not include the background z-level. If the 
background z-level is required, it wiU be explicitly referred to as such. 

7.1. Intermediates 

During the compositing process, access to an intermediate composite buffer is 
required. This buffer may be large enough to hold any intermediate compositing results 
for a run of maximal size - Le. the width of the output device. One skiUed in the art wiU 
see that the composite buffer can be of any size, and may thus require repeated 
applications of the described processes. 

The compositing module 612 also makes use of a work buffer, which is large 
enough to store any intermediate compositing results for a single pixel. These buffers 
may be formed within an ASIC, for hardware implementation, or alternatively in general 
memory for software implementations, and typically have red. green, blue and alpha 
(transparency) components. 

7.2. Z-Ievel Fills 

Each z-level contains a reference to ay?//, which may be one of the following 

types. 

A solid fill is one where the color values and the alpha values remain constant 
over the length of the run. Examples of soUd fills are an opaque red fill, or a blue fill with 
a 50% alpha. 

An x-dependentfill is one where the color values are not necessarily constant over 
the nin of pixels. In this definition, an x-dependent fill has a constant alpha value over 
the run. Examples of an x-dependent fill include a Unear or radial blend with the same 
alpha at aU blend control points, or a bitmq) texture fill (with a constant alpha). 

A variable alpha fill is one where the alpha can vary over the run. Examples of a 
variable alpha fill include a linear or radial blend where alpha is not the same at each 
blend control pomt. or a bitmap texture fill with a non-constant alpha. 

7.3. Basic Flow 

The basic flow of the compositing algorithm is shown in Fig. 13. This is a high- 
level overview, and a more detailed description for some steps will be provided later. 

The first step 110 is to test to see if the input set is empty. If the set is empty, this 
means that there are no fills present, and it is the background fill that should be drawn, 
this being generated at step 111. The compositing process is then finished. 
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If the input set is non-empty, then the next step 112 is to determine whether the 
input ordered set of z-levels contains any fiUs that are classified as variable alpha In the 
compositing algorithm, it is the topmost variable alpha fill that is of interest. Tins check 
is performed by iterating through the ordered set of input z-levels one by one, testing for a 
variable alpha fill. The preferred order is to iterate through the ordered set such that the 
first element checked is the topmost fill. The check terminates when the first variable 
alpha fiU has been found, or the entire set has been checked and no variable alpha fill was 
found. 

The next step 113 performs a contribution calculation on all fills that are above the 
topmost variable alpha fill that was discovered in step 112. If no variable alpha fills are 
present, then this contribution calculation is performed on all of the fills present in the 
input ordered set. 

A decision 114 is the next step. If variable alpha fiUs were discovered in step 112, 
then execution moves to step 115. Oflierwise, execution moves to 116. 

In the case that there are no variable alpha fills present, step 116 initializes the 
composite buffer as being empty. 

Step 115 performs a bottom-up composite of all fills below and including the 
topmost variable alpha fill. Next, in step 117. the total contribution of aU the fills bottom- 
up composited in step 115 is calculated. 

The step 118 uses a top-down composite to composite all fills above the topmost 
variable alpha fill. 

The final step 119 produces the output pixels as suitable for presentation on the 
output display device. 
7.4. Graphical Overview 

The stages of the processing are illustrated graphically in Figs. 9(a) and 9(b) and 
Fig. 10. These figures demonstrate different compositing scenarios. 

Fig. 9(a) shows a simple example. An ordered set of input fills 70, 71, 72 is 
shown. Each of these fills is solid. The contribution calculation is perfo'ime'd. and 
respective example conlributions are shown at 67. 68 and 69. There are no variable alpha 
fiUs present, so the next process is the top-down compositing. The results are stored in 
the work buffer 73, which is then repUcated the necessary number of times to generate the 
required output pixel run 74. hi this example, a run of four pixels is produced. 
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The second scenario Kg. 9(b) involves an x-dependent fill. Fills 75 and 77 are 
both solid, while fiU 76 is x-dependent The contribution calculation is performed, and 
the results are shown by 80. 81 and 82. Since there are no variable alpha fills present, the 
next process performed is the top-down composite. The pixels for the x-dependent fill 76 
are generated and composited into the composite buffer 79. while the color for the soUd 
fills 75 and 77 is generated and composited into the work buffer 78. The contributions of 
both the work buffer and the composite buffer are shown. The final step is to produce the 
output pixels by compositing the single pixel of the work buffer into each pixel of the 
composite buffer, generating the output pixels 83. 

The final example of Fig. 10, is more complex, and includes a variable alpha fill. 
FUl 84 is a soUd fill, fill 85 is an x-dependent fill, fill 86 is a variable alpha fill, and fill 87 
is a soUd fill. After the contribution calculation, fills 84 and 85 have contribution values 
associated with them (as shown by 88 and 89 respectively), and everything below and 
including the variable alpha fill 86 does not. The next process to be executed is the 
bottom up compositing. Fills 86 and 87 are bottom-up composited, and the results stored 
in the composite buffer 92 (cl). After flie bottom up composite, it is possible to work out 
the contiibution of the results stored in the composite buffer, by simply subtracting the 
contributions, of fills 88 and 89 fi-om 100%. The next step is the top-down composite. 
The topmost fill 84 is soUd, and so is composited into the work buffer 95. Fill 85 is x- 
dependent, and so is composited into the composite buffer 97 (c2). After the top-down 
composite step, it can be seen tiiat the work buffer 95 contains tiie compositing results for 
a single fill 84, and the composite buffer 97 (c2) contains tiie compositing results for 
fills 85, 86 and 87. The final stage is to generate tiie output pixels. The single pixel of 
tiie work buffer is composited into each pixel of tiie composite buffer to generate flie 
output pixels 96. 

7.5. Contribation Calculation 

The step 113 of Fig. 13 is examined in more detail wifli reference to Fig. 11. 
Recall tiiat tiiis process is performed on aU fills in tiie ordered set of input z-levels tiiat are 
above tiie topmost variable alpha fill as found in step 112. If no variable alpha fill exists 
in tiie input set, then tiiis process is performed on all fills in tiie input set. If tiie topmost 
variable alpha fill is also tiie topmost fill in tiie input set. tiien tiie contiibution calculation 
does not need to be performed. 
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The contribution calculation introduces a new variable for each fill that is to be 
subject to this process. Thisisthecontribution^valueof afiU. This is always initialized 
to represent a contribution of zero. For exan^le, it is possible to use an 8-bit integer to 
store the contribution, with 0 representing zero contribution, and 255 representing lOQo/o 
contidbution. The purpose of the contiibution calculation is to determine how much each 
fill contributes to the ou^ut pixels. 

Two variables, accumulated.opacity and coverage are introduced here. 
Coverage is a description of which spel elements of the A-buffer associated wilh the fill 
being processed are currently under consideration. It is convenient to express coverage 
as an A-buflfer. AccumnIated_opacity is a measurement of the extent that the spels 
under consideration are obscured by fills above the current fill. In the following 
discussion, an accumulated.opacity of zero percent refers to fiill transparency, and an 
accumulated_opacily of one hundred percent refers to fiUl opacity. These two variables 
along with an indication of the current.flll bemg processed, form a set of parameters that 
describe the state of the contribution calculation algorithm. The concept of a stack, as 
known in the art, is used to store sets of these three parameters that can be read in by the 
contiibution calculation algorithm and acted upon. In Ihe following discussion, the term 
push means to add a set of parameters to the top of the stack, and^op means that the set 
of parameters on top of flae stack is removed. The stack wiU be referred to as the 
Parameter Stack. 

Fig. 48 shows the effect of accumulated_opacity in an A-buffer when rendering, 
from top to bottom in order. Portions of objects A 4800. B 4801, C 4802 and D 4803 
each have an opacity value applicable to certain spels in the corresponding A-buffer. It is 
noted that an opacity of 100% means tiiat the object is fidly opaque. The objects are 
being rendered over a background object 4804 having 0% opacity (ie. transparent), 
hi Fig. 48. the black shading indicates coverage by the object over the relevant subpixel 
of the A-buffer. Initially, applying the object A 4800 to the background 4804 reveals a 
result 4805. Since only two spels of A are 35% opaque, the contribution of object A 4800 
is only 17.5% to the resultant pixel value. Note tiiat 0% opacity gives 0% contiibution 
The accumulated opacity resolves as illusti:ated to a tree where the number of branches is 
Umited to the number of subpixels in the mask, in this case four. Two results 4806 and 
4807 of the appUcation of object A 4800 to the background 4804 are shown. Next when 
object B 4801 is appUed, such only operates on the lower two spels of the A-buffer. As 
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the lower left spel 4808 intersects with that of object A. the two contributions wiU 
accumulate, whereas for the lower right spel 4809, object A 4800 had no contribution so 
the result at this stage for that spel will be limited to that of object B 4801. The net 
contributions of object B are detennined to be 16.5% from the calculations shown. 
Accordingly, after applying object B. the spels have respective opacity levels of 61% at 
4810. 35% at 4811. 40% at 4812 and 0% at 4813. Object C 4802 has opacity = 100% and 
impacts on the right two spels giving results 4814, 4815, 4816 and 4817 thereby 
indicating a contribution of 40% to the pixel. Since object C is foUy opaque, it is not 
necessary to consider any lower objects (object D) where such may intersect with 
object C. Hence, for the right spels, the tree branches stop at object C. Object D 4803 
has 100% opacity and as seen at 4818, 4819, 4820 and 4821 provides a 26% contribution 
to the pixel. It is seen from Fig. 48 that the accumulated contributions detennined from 
the opacity values for ttie active graphic objects sum to 100%. 

For the A-buffer of Rg. 48. the resulting contributions may be expressed as 
follows: 



A = 35% 


A = 0% 


B = 0% 


B = 0% 


C = 0% 


C = 100% 


D = 65% 


D = 0% 


A = 35% 


A = 0% 


B = 26% 


B = 40% 


C = 0% 


C = 60% 


D = 39% 


D = 0% 



From Kg. 48 it is ^parent that a tree is traversed in which the number of levels is 
determined by the size of the sub-pixel mask (A-buffer). Further as seen with respect to 
the object C 4802, traversal of any branch of the tree can be halted as soon as 100% 
opacity is reached, as no fiirther contribution may be made to that sub-pixel. 

Returning now to the actual processing, the first step 98 of the process of Fig. 11 
is to initialize the parameters. Current_flll is initiaUzed to be the topmost fiU in the input 
set. Accumulated_opacity is initiaUzed to zero percent, and coverage is initiaUzed to 
represent total coverage of the A-buffer. This initial set of parameters is pushed onto the 
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Parameter Stack. In addition, aU the contribntion_vaInes of aU the fills that are to be 
processed are initialized to zero. 

An initial step 99 of a main process loop pops the next set of parameters from the 
Parameter Stack. This is checked for an empty result in step 100. If it is the case that no 
more parameter sets exist on the stack, then the contribution calculation ends. If a 
parameter set was successfully popped from the stack, the next step 101 is to calculate the 
contribution of the cnrrent_fill given tiie current accumnlated.opacity and coverage, 
and add this to total contribution_value of the current.fiU. The calculation is 
performed according to the following equation: 

current_contrib = cuirent_alpha * (1 - accumulated_opacity) * coverage_ratio 

where 

current_alpha = the alpha value of current_fiD 
accumulated_opacity = the accumuIated_opacity as given in the 

current parameter set 
coverage_ratio = active_ratio(coveragen A -buffer) 
A-bu£fer = the A-bufifer associated with the current_fiU, and 
Active_ratio(A.bufiFer) = ratio of the number of active spels in the given A- 
buffer, over the total number of spels in the given A-buffa-. 

Note that aU elements of the main equation are intended as values expressed as a 
real number with a range [0..1]. This is for simpHcity of expression of this equation, and 
it will be clear to one skilled in the art how to manipulate the equation in order to 
accommodate other means of expression of these elements. In the case of 
current_coverage ratio, the [0..1] range represents the ratio of the number of 'active' 
spels in the intersection of the coverage A-bu£fer with the current_fill's A-buffer versus 
Ihe total nimiber of spels in an A-bufifer. 

This calculated c«rrent_contrib value is then added to the contribution_value of 
the current_fill. 

The next step 102 of the process of Fig. 11 checks to see if there is a next fill to 
process. If the current.fUl is either the last fill in the input set, or if the next fill is a 
variable alpha fill, this check will have a negative result. Execution of the process then 
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retuxBs to step 99. If the check result is positive, then the current fiU is updated to 
identify this next fill. 

In th. ne« step 103. a ^ of parameters is pushed onto the Parameter Staok 
These paiiameters are: 

cuirent_fill <~ current_fill 
accumulated_opacity <- accumulated_opacity 
coverage <- coveragefl (coverage fl A - buffer) 
The next step 104 updates the current parameter set as per: 
current_fill <- current_fill 

accumulated.opacity ^ (l - cuirent_alpha)* (1 - accumulated_opacity) 
coverage <- coverage 0 A - buffer 

Note that accumulated_opacity and current.alpha are intended as values 
wqpressed as a real number with a range [0.. 1]. 

A graphical indication of the operations appUed to coverage can be seen m 
Figs. 12(a) to 12(d). An A-buffer 107 provides an «cample representing coverage A 
second example A*ufier 106 represents the A-bnffer of a fill. When the operation k> 
generate a new coverage A-bniler as given in step 103 is applied, the result is the A- 
buffer 108. When the operation to generate a new coverage A*„»r given in step 104 is 
15 ^plied, the result is the A-buffer 109. 

Once this update has h^pened, accumnlated.opacity is tested to see if a 
sufficient degree of opaqueness has been achieved. An example threshold is 255/256 of 
fUll opaqueness. If sufficient opaqueness has been reached, then execution of the process 
moves back to step 99. Otherwise, execution of the process continues fiom step 101. 
20 7.6. Bottom Up Composite 

Referring back to Fig. 13. if variable alpha fills have been detected in the input set 
m step 114, then the bottom up compositing step 115 will be performed. 

The bottom up compositing process composites fills in order fix,m the bottom- 
most up to and including the topmost variable alpha fill. When the following discussion 
25 mentions a 'next' fill, this refers to the fill immediately above the current.fMI. 

The bottom up composite process requires access to ^fill generation buffer. This 
buffer must be able to store at least as many pixels as the composite buffer. For example 
t^s buffer may be realized by utilizing half of the composite buffer. This halves the sizi 
of the composite buffer, and in the case of an output run that is longer than half the 
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ori^al lengO. of tte con^,. buffer, a «coo^ .ppUcation of fl.. 
module «12 wiU be required. lung 

Referring to Fig. 6. fte bottom compositmgp™«ss is d^icted in which a first 
step SO .s to composite the background ffll into the composite buffer. Because Ms is 
guaranteed to be the firs, fin composited into composite buffer, no special calcuIaHon 
■srequrred. H.. fiU color is just copied ir. For «.e purposes of the fir« execution of the 
next step, the current.ffll is considered to be the background fill. 

lie next sh=p 51, is a test to see if fljere is a next fiU that requires bottom up 
-mpo^tag. If the crrent.M is .he ,„pn.„s. variabie alpha fill, then there are no 
fteher fills to bottom-up composite, and the bottom up c«nposito precess moves to the 
pesnultimate step 55. 

The detennining of the contribution of the bottom up composite result at step 55 is 
perfonned according to the foUowing algorithm: 

15 total_contrib = 0 

for each fiU above topmost variable alpha fill 

total_contrib = total_contrib + fill[n].contrib 
bottom_up_contrib = 1 - total_contrib 

where 

20 fill[n].conlrib - contribution value of the n'th fill above the topmost 

varwble alpha fill, as calculated in step 113 of the flowchart He. 13. 

Once flae contribution value for the bottom up composite result has been 



where 

30 



for each pixel in conq)osite buffer 

cb[n] « cb[n] * bottom_up_contrib 



cb[n] « n'th pixel of composite buffer. 
The above expression of a scalar vahe (b„«„m_up_con.rib) multiplied by a 
CO or value (cbfn,) rei^ ^ a multiplicatiou by the scalar of all the component, of L 
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If Step 51 detennines that there are further fills to be bottom up composited, then 
the current_fm is set to be the next fill via step 52. The current_flll is then tested to see 
if it is an x-dependent fill in step 53. If the fill is not x-dependent, then the fill color is 
generated into the work buffer according to step 54. The contents of the work buffer are 
then repeatedly composited into the composite buffer using the foUowing algorithm: 



for each pixel in composite_buffer 

cb[n] = cb[n] + work_alpha * (cb[n] - work) 

where 

10 cb[n] = n'th pixel of composite buffer, 

work_alpha = the alpha of the pixel in the work buffer, 
work = the pixel in the work buffer 

Execution of the bottom up composite then returns to step 51. 
15 If the fill is x-dependent, then all pixels for the run are generated in step 56 into 

the fiU generation buffer. The fill generation buffer is then composited into the composite 
buffer in step 57 using the following algorithm: 

for each pixel in composite and fill buffers 
20 cb[n] = cb[n] + fb[n].alpha * (cb[n] - fb[n]) Eqn. 1 

where 

cb[n] = n'th pixel of composite buffer 
fb[n] = n'th pixel of fill buffer 

fb[n].alpha = the alpha of the n'th pixel in the fill buffer. 

25 

Execution of the bottom up composite then continues to 51, 
7.7. Top Down Composite 

The top-down compositing process, shown at step 118 in Fig. 13, is now 
described in detail with reference to Fig. 55. It is appUed to all the fills above the topmost 
30 variable alpha fill as determined in step 112. If no variable alpha fills are present in the 
input set, then the top down alpha process is appUed to every fill in the input set. 

The top down process utilizes the contribution_values for the fills as calculated 
in step 113. It should be noted that in the top down compositing process, the order in 
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which the fills are processed is unimportant in tenns of the final result. In the exanq)le 
below, the fills are processed in the same order as they q)pear in the input set. beginning 
with the topmost fill in the set. 

As seen in Kg. 55, the first step 601 of the top down compositing process is to 
detennine whether there are further fills to be processed. If there is no next fiU to process, 
or the next fiU is the topmost variable alpha fill as determined in step 112, then the 
process moves to step 607. 

If there are fiirther fills to be processed, the current fiU is stored in a variable 
current.mi in step 602. The next step 604 is to check the contribution_vaIue of the 
current_£m to see if it is greater than zero. If this test returns false, execution moves to 
step 601. 

If the contribution_value is greater than zero, execution moves to step 603. 
Step 603 tests the current_fiU as to whether it describes an x-dependent fill. If the fiU is 
not x-dependent, execution moves to step 606. Otherwise, if step 603 returns a positive 
result, execution moves to step 605. 

A non x-dependent fill is handled in step 606, proceeding as follows. The color 
for the fill is generated, and composited into the work buffer, according to the following 
algorithm: 

wb = wb + current_fiU.contrib * current_fill.color Eqn. 2 

where 

wb = work buffer, 

current_fill.contrib = contribution_value of current.fill, 
current_fill.color = color value of current_fill. 

An x-dependent fill is handled in step 605, proceeding as follows. As each pixel 
of the x-dependent fill is generated, it is immediately composited into the composite 
buffer, according to the following algorithm: 

cb[n] = cb[n] + current_fill.contrib * gp 
where 

cb[n] = n'th pixel of composite buffer, 

Current_fill.contrib = contribution_value of current_fill, and 
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gp - generated fill pixel cuirentiy under consideration. 

After either step 605 or step 606 has completed, execution moves back to 
step 601. 

If step 601 determines that there are no more fills left to process, execution moves 
to step 607. In this step, the work buffer is composited into the composite buffer, to 
produce the final result. This step is performed according to the following algorithm: 

for each pixel in composite__buffer 
cb[n3 = cb[n] + wb 

where 

cb[n] = n'th pixel of composite_bufrer, 
wb = work buffer. 



The top down composite is now complete, and the composite.buirer contams the 
fiilly composited pixels of the output run. 
7.8. Alternative Compositing Approaches 

An alternative ^proach of the compositing process is described below. This 
alternative eliminates the need to do potentially large bottom up composites when a 
variable alpha layer is encountered, instead using the top-down compositing process 
wherever possible, thus maximizing the benefits that top-down compositing provides. 

The contribution calculation process, as described above, stops when it encounters 
a topmost variable alpha layer. The alternative compositing method checks to see if there 
is a layer existing below this topmost variable alpha layer. If so, then the compositor is 
invoked recursively, with this layer below the topmost variable alpha layer being the 
topmost layer in the new invocation. This new invocation, and any subsequent 
invocations, process the layers as normal, and when they reach fijll opacity, pixels are 
generated into the work and composite buffers as normal. When any recursive invocation 
returns to the parent, the result in the composite buffer is then composited with the 
variable alpha layer discovered in the parent. The contribution of the results in the work 
buffer and composite buffer are then calculated as in step 55 of Fig. 6, and the pixel 
values m both these buffers are adjusted accordingly. 
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One generalization of these compositing q>proaclies is as follows, for a run of 

pixels: 

(i) divide the layers into groups, with each group being separated by a 
variable-alpha layer; 

5 (ii) perform a top-down composite for each group to determine a group pixel 

value, thereby resolving the original layers into a reduced number 
comprising group pixel values separated by the variable alpha layers; 

(iii) where the contribution of any one group is within a predetermined 
threshold of 100% (ie. fully opaque), all variable alpha layers and groups 

1^ beneath that threshold group can be ignored as non-contributing; and 

(iv) for each pixel in the run, perform a bottom-up composite of those 
contributing groups and variable alpha layers to determine the 
corresponding pixel value. 

A further implementation of the compositing processes can be accompUshed using 
15 two buffers in the following fashion: 

(i) commencing with the topmost layer and using a first buffer, perform a top- 
down composite down to and including the first variable alpha layer; 

(ii) using a second buffer, perform a top-down composite of those layers down 
to and including the next variable alpha layer; 

20 (iii) composite the first buffer over the second buffer and retain the result in the 

first buffer; and 

(iv) repeat steps (ii) and (iii) until all layers have been composited. As before, 
where the contiibution of any one group of layers above a variable alpha 
layer reaches is within a predetermined fljreshold of 100% (ie. ahnost or 
opaque), all variable alpha layers and groups beneath that threshold 
group can be ignored as non-contributing and compositing can cease. 
It is noted that the further implementation described above will not accurately 
depict the image as the progressive inclusion of the variable alpha layers incorrectly 
accounts for the contiibutions of layers &om lower groups. For example, if the 
30 contiibution of all top layers down to but not including the first variable alpha layer is 
80%, then the contiibution of the first variable alpha layer and all lower layers should be 
20%. Compositing the first variable alpha layer with those above will distort the balance 
of contibutions to give inaccurate, but faster compositing. Whilst such may not be 
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desireable for final image rendering, such may be sufficient for previewing stages of 
image construction where accurate detail of all layers is not necessary. 

The predetermined threshold is typically dependent iq)on a desired precision of the 
composite result and the number of operations to be performed. Such may for example be 
within a least significant bit of a color channel, which for 8-bit color gives a contribution 
of 255/256 (99.6%). Other thresholds may be chosen. 

Fig. 61(a) illustrates a generalization that can be performed for a top-down 
composite of a number of layers within a run of pixels. As shown, a number of 
layers 6101-6107 are shown which have different types. One type is a constant color 
level which, by its very nature, can include a constant alpha conq)onent. These are 
levels 6101, 6102, 6104, and 6106. Another type of level is a variable color level but 
having constant alpha. An exanq>le of this is a linear blend that occurs over the run of 
pixels. Again, because the alpha value is constant, it may be considered as a single 
variable color and examples of this are the levels 6103 and 6105. The final type of level, 
an example of which is level 6107. is a level which includes both variable color and 
variable alpha. By virtue of its nature, this level may be generalized as a variable alpha 
level and operates to change the contribution of aU levels below that level in the rendering 
of the span of pixels. 

The generalization of Fig. 61(a) is that with a top-down composite, all levels 
above a variable alpha level, each of which having constant alpha, may be top-down 
composited to provide a single color level which, depending on the original source levels, 
may be a variable color level. This generaHzation is shown by virtue of a new color level 
6108 representing a top-down composite of the levels 6101-6106. The level 6108 is 
placed above the previous variable alpha level 6107. 

Fig. 61(a) as such indicates feat any number of levels having a constant alpha 
component may be top-down composited to provide a single level (eg: 6108) of constant 
alpha which may be then composited over a level having a variable alpha. 

Fig. 61(b) shows how this generalization can be appUed to a compositing stack 
incorporating numerous layers in which groups 6110, 6112, and 6114 of layers of variable 
color are each separated by layers 6111 and 6113 of variable alpha. In such an 
arrangement, each of the groups of layers having variable color constant alpha can be top- 
down composited to produce a corresponding single layer of constant alpha, these being 
illustrated as new layers 6115, 6116, and 6117. Each of these new layers can then be 
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composited with their respective interposed variable alpha layers 6111 and 6113 in a 
bottom-up composite to provide an accurate composite of the run of pixels. Most 
significantly, by dividing the compositing operations into as many top-down composites 
as possible, the number of compositing operations are minimized over the run of pixels. 
5 In this regard, because of the operation of any variable alpha level, such necessitates a 
composite at each pixel in the run of pixels. As such, the combination of top-down 
composites followed by a bottom-up composite provides optimizations previously not 
available. 

An extension of this principle is shown in Fig. 62 where a compositing stack 6200 
10 for a span of pixels is fonned using various layers 6201-6212 which include combinations 
of constant color, variable color and variable alpha layers. According to the variation, the 
compositing stack 6200 is composited using only top-down composite operations to 
provide pixel values for the run of pixels. Essentially, the stack 6200 is divided into 
sections, each delineated by a variable alpha layer. In this arrangement a top-down 
composite 6213 is performed on the first group of layers up to and including the first 
variable alpha layer, this groiq) being the layers 6201-6204. The result of the composite 
is retained in a first buffer 6214. A second top-down composite 6215 is then performed 
on the next group of layers up to and including the next variable alpha layer, this group 
being the layers 6205-6207. The result of the composite 6215 is then stored in a second 
20 buffer 6216. The composite results contamed in the buffers are then composited together 
with the first buffer 6214 being composited over the second buffer 6216, and the result of 
that composite being written back into the first buffer as indicated at 6217 in Kg. 62. The 
next group of layers 6208-6211 is then top-down composited 6218 with again the 
composite value being stored in the second buffer 6219. Again, the first buffer 6217 is 
25 composited over the second buffer 6219 to give a result 6220 stored in the first buffer. 
Finally, in flie example of Fig. 62, the last group of layers, which includes only the 
constant color layer 6212 is then top-down composited 6221. In this case, since the group 
of layers contains only a single constant color, the composite operation is effectively a 
null and the value of the layer 6212 is copied into the second buffer 6222. Again, the 
30 buffers are composited together with the final result 6223 being retained in the first 
bviffer. 

It will be appreciated firom the description of Fig. 62, that reUance upon top-down 
compositing only does not necessarily guarantee that the final composite result will be 
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entirely accurate for the particular span of pixels. However, the present inventors have 
determined that a desired level of precision may be estabUshed for the compositing 
operation such that the final composite result can be created to a detenninable level of 
accuracy. Such has impUcations where very fast compositing is necessary, possibly at the 
expense of image quaUty. Examples of tbis may be in previewing the image creation 
process, for example as performed by a graphic artist. Other implementations may be 
where the compositing process is able to be performed on a number of different platforms 
of varying processing capacity. Particularly, where a compositing time requirement is 
specified (eg: in real time operations) the complexity of the compositing stack may be 
used to trigger a particular type of composite (eg: Fig. 61(a) -accurate but slower, or 
Fig. 62 - faster but possibly less accurate) in order to meet system requirements. 

A further alternative approach is shown in Fig. 64 and provides an unprovement 
upon the previously described arrangements by removing the need for bottom-up 
compositing. The exclusive use of top-down compositing allows for increased 
15 exploitation of the benefit of top-down compositing, being early exit i&om the process 
when opacity is sufificientiy close to 100% resolved. The cost of this alternate approach, 
in comparison to that of Figs. 9 and 62, is an additional work buffer (here called "Work 
Buffer 2"). This new work buffer has the same elements as Work Buffer l(eg. 78), but 
each element is an rgbo instead of rgb (r= red, g=green , b=blue, and o=opacity). 
20 In operation, the compositing stack is partitioned into sections separated by 

variable opacity layers. Work Buffer 0 (eg. 73) and Work Buffer 1 are used on the 
uppermost section as with the previous implementation. Work Buffer 1 is then 
composited over the first variable opacity layer and the result stored in Work Buffer 2. 
Work Buffers 0 and 1 are then appUed to the next lowest section in the same feshion as 
25 they were appUed to the topmost section, except that at the conclusion: 

(a) Work buffer 2 over Work Buffer 1 is stored in Work Buffer 2, and 

(b) Work Buffer 2 over the next variable opacity layer is stored m Work 
Buffer 2. 

With reference to the compositing ^proach 6400 shown in Fig. 64, step 6401 
30 composites fixed colors firom the compositing stack to form a fixed color in Work 
Buffer 0. Step 6402 composites two variable colors into Work Buffer 1. Step 6403 
composites Woric Buffer 0 over Work Buffer 1 and stores the result in Work Buffer 2. 
These steps essentially correspond with those of the previous approach. In step 6404, 
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Work Buffer 1 is composited over a first variable opacity layer with the result being 
stored in Work Buffer 2. Step 6405 operates in the previous fashion to composite the 
next two fixed colors, expect that their contributions are retained locally within Work 
Buffer 0. Step 6406 then composites Work Buffer 2 over Work Buffer 0 with the result 
being stored in Work Buffer 2. Step 6407 the composites Work Buffer 2. which is a 
variable opacity color over the next variable opacity layer with the result be stored in 
Work Buffer 2. Step 6408 - 6410 effectively repeat steps 6401-6403. Step 6411 the 
composites Work Buffer 2 over Work Buffer 1 with the result again being retained in 
work Buffer 2. Step 6412 completes the compositing process by processing Work 
Buffer 2 over the next variable opacity layer, the final result be retained in Work Buffer 2. 

As such, this approach is seen to accommodate variable opacity layers whilst 
maintaining a top-down traversal of the compositing stack. Early exit from the method 
6400 can be achieved as described earlier when the accumulated opacity reaches a 
predefined threshold. In calculating accumulated opacity the opacity contribution of 
variable opacity layers can safely be taken as zero. Alternatively, the opacity contribution 
of each variable opacity layer may be assumed to be a minimum value, for example 
obtained by a search for a pixel within the variable opacity layer having a minimum 
opacity. A finther extension of this principle is to track the opacity contribution across 
the whole composition of the stack as a vector (eg. on a per-pixel basis), such that the 
accumulated opacity of pixels will reach the predefined threshold on an individual basis 
thereby causing early exit. 
7.9 Top-Down Advantages 

The top-down compositing techniques described above are preferred over the 
bottom-up techniques. The top-down techniques are more efficient in the amount of 
calculation required to composite two layers together. This can be seen with reference to 
the Eqns. 1 and 2 presented in Sections 7.6 and 7.7. 

Top-down compositing also has the significant advantage of allowing the 
compositing process to terminate as soon as it is determined that subsequent compositing 
operations will have no visible effect on the output. This "early termination" happens 
very often in practice, and results in a performance increase. 
8. PIXEL GENERATION 

The Pixel Generation Module 611 of Rg. 56 is concerned with generating a run of 
pixel values for a single Z-level. 
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The Compositing Module 612 combines color values of z-level fills in order to 
generate ou^ut pixels. The source color values that the compositing module 612 
combines are either expUcitly defined or must be generated from some form of fiU 
description. Z-levels of soUd color fill expUcitly define a source color value that is ready 
for compositing. Z-levels of complex fill type, such as gradients (linear or radial) and 
bitmap image fills, require source color values to be generated in a manner that is 
dependent upon the raster space location of the pixel to generate. This process of 
detennining the source color values for a z-level of complex fill is referred to as pixel 
generation, performed by the Pixel generation Module 611. 

A processing flow of the Pixel Generation Module 611 is outlined in Fig. 39. As 
input, the module 611 is provided at step 393 with a start and end location in raster space, 
along with a reference to a complex Z-level containing the required fill description.' 
Depending on the fill type of the Z-level. detemiined in steps 394 and 395. the compositor 
will generate a run of color values for either: 
15 - a Linear Gradient in step 396; 

- a Radial Gradient in step 397; and 

- a Bitmap Fill in step 398. 

Generated color values are stored into a compositing buffer or a fill generation 
buffer ready for compositing by the compositing module 612. Details of color generation 
20 for each fill type are now given. 

8.1. Linear Gradient Pixel Generation 

A z-level may have a linear gradient fill. In the art, linear gradient fills are often 
referred to as linear blends. A linear gradient fill involves shading a surface using color 
values that depend on the perpendicular distance of the point to be shaded from a 
25 specified imaginary line. 

Linear gradients may be described using a gradient look-up table. A gradient 
look-up table defines color values for a gradient over some range of index values. A 
gradient look-up table may be implemented as an ordered set of one or more gradient 
look-up table entries. Each gradient look-i^ table entry comprises an index value and a 
corresponding color value. Gradient look-up table entries are ordered by index value 
from lowest to highest. 

Referring to Fig. 25, a gradient look-up table 260 is shown. The index value 
range of the gradient look-up table 260 can be seen to be from 0 to 255 inclusive. The 
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gradient look-up table 260 conqjiises two gradient look-up table entries 261 and 262. The 
first of the gradient look-up table entries 261 defines a white color value at an index value 
of 63. The second of the gradient look-up table entries 262 defines a black color value at 
an index value of 191. 

For an index value between two entries of a gradient look-up table, a 
corresponding color value may be calculated using linear interpolation of the color values 
of the two surrounding gradient look-up table entries. For an index value less than the 
minimum index value of aU gradient look-up table entries, or for an index value greater 
than the maximum index value of all gradient look-up table entries, a corresponding color 
value may be determined by directly using the color value of the gradient look-iq, table 
entry with closest index. 

For example, in Fig. 25, the color value corresponding to an index value between 
the two look-up table entries 261 and 262 is detennined using linear interpolation of the 
white and black color values of the entries respectively. Continuing the example, for an 
index value below the index value of the first look-up table entry 261, say index value 50, 
the associated color value is that of the nearest look-up table entry (hence a white color 
value). Conversely, for an index value above the index value of the last look-up table 
entry 262. say index value 255, the associated color value is that of the nearest look-up 
table entry (hence a black color value). 

When generating a raster pixel color value &om a gradient fill, it is necessary to 
calculate an index into a gradient look-up table corresponding to the location of the pixel. 
To do this, a transform of the pixel's raster space coordinate location into a gradient space 
is performed. In order to map the gradient space into the look-up table index range, it is 
useful to define a gradient space surface. Each point in a gradient space surface m^l to a 
particular gradient look-up table index. A raster pixel's color value may be detennined 
using the index value and a gradient look-up table. 

For a linear gradient, the gradient space surface may map a point to an index value 
in a manner that depends only on the horizontal position of the point within the gradient 
space surface. This means that for a linear gradient, there is no need to calculate any 
vertical component of the position in gradient space. In order to change the appearance of 
a linear gradient in raster space, it may be desired to modify parameters such as the • 
gradient's position, scale, and rotation. To achieve such effects, a tr^sform of the linear 



wo 2004/114223 



-92- 



PCT/AU2004/000842 



gradient space surface into raster space using a gradient to raster space transform may be 
perfonned. 

Fig. 25 also shows an example of a square gradient space surface 263 of 
width 32768 centred at the gradient space origin. Color values across this gradient space 
suifece are shown using the technique for linear gradients described above, whereby, the 
color value only depends on the horizontal gradient space position. Fig. 25 iUustrates this 
gradient space surfece 263 being mapped onto a raster space device 264 for output. It can 
be seen that this mapping involves some rotation and scaling of the gradient space 
surface. 

Linear gradient fiUs may be rendered by incrementally tracking an index into a 
gradient look-up table. Such a technique involves pre-calculating three parameters that 
depend on the gradient to raster space transform. Referring to Fig. 32. given a linear 
gradient description comprising a gradient look-up table and a gradient to raster space 
transform, before calculating any ou^ut pixel color values, it is possible to calculate at 
step 311: 

- the change in gradient space horizontal position per pixel step in raster 

space horizontal direction— ' 

dx 

- the change in gradient space horizontal position per pixel step in raster 

space vertical direction — ; and 

dy 

- the gradient space position, u„ , corresponding to the raster space origin. 
These pre-calculated values are dependent upon the gradient to raster space 

transform and only need recalculation when the transform is modified. 

A graphical representation of these gradient tracking parameters may be seen in 
Fig. 27. In Fig. 27, a display of a raster output device of raster space X-Y 269 may be 
seen. A gradient defined in gradient space U-V 270 is rasterized into the raster device 
space 269. A raster scan line 267 of the raster device is to contain a run of gradient filled 
pixels commencing at a start pixel 268 within the scan line. For a single raster pixel step 
in the positive horizontal raster space direction, the corresponding change in gradient 
space horizontal position, ^ 271. is shown. For a single raster pixel step m the positive 
vertical raster space direction, the corresponding change in gradient space horizontal 
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position, — 273, is shown. In the figure, similar parameters 272, 274 corresponding to 

the vertical gradient space position are also shown. Calculation of these values is not 
required for linear gradirats. 

It is thus possible to proceed to generate linear gradient pixel color values for one 
or more contiguous pixels forming a run to be output to the fill generation buffer. Given 
the raster space coordinate location (x,^^,y,^^) of the left-most raster pixel in a run to be 
generated, it is necessary to first initialize the gradient space horizontal position, u at 
step 312 in Fig. 32. In Fig. 27, an example of the left-most raster pixel in a run to be 
generated is indicated at 268. The initial u value may be calculated using the following 
formula: 



It is therefore possible to determine an actual index into a gradient look-up table 
by directly using the u value. Using such an index value, a corresponding color value 
from a gradient look-up table can be determined as previously described in step 313. 

After generating a pixel's color value, if there are still more pixel color values in 
the run to generate from the linear gradient, as determined in step 310, the gradient space 
horizontal position value, « may be incremented in step 315 according to the following 
recurrence relation: 

ax 

This process repeats until aU required pixels in a run have been generated. 
Following this, if there are more runs requiring rendering using the same linear gradient 
as determined at step 314, then the same process may be repeated by firstly recalculating 
the initial gradient space horizontal position, u, at step 312 for the start pixel of the new 
run to render and then .repeating the process at step 313 of generating pixel color values 
for each pixel of the run. 
8.2. Radial Gradient Pixel Generation 

A z-level may have a radial gradient fill. Radial gradient fills are also referred to 
as radial blends. A radial gradient fill can be thought of as shading a surface using a color 
value that depends on the distance of the point to be shaded from a specified centre point 
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Of the radial gradient. The process of generating radial gradient pixels is similar to the 
process of generating linear gradient pixels as previously described. 

As with linear gradients, radial gradients may be described using a gradient look- 
up table implemented as an ordered set of one or more gradient look-up table entries. 
5 Radial gradients also make use of a gradient space surface. Again, each point in a 
gradient space surface maps to a particular gradient look-up table index. Again, the color 
values of two gradient look-iq, table entries may be linearly interpolated to obtain a color 
for an index that isn't ejq)licitly defined in the look-up table. 

For a radial gradient, the gradient space surface maps a point to an index value in 
10 a manner that depends on the distance of the point fi-om the centre point of the gradient 
space surface, hi order to change the appearance of a radial gradient in raster space, 
parameters may be modified, such as the centre point and the scale of the radial gradient. 
To achieve such effects, the radial gradient space surface may be transformed into raster 
space using a gradient to raster space transform. 

Rendering radial gradients in the traditional fashion is computationaUy expensive. 
Typically, it would be necessary to transfomi each pixel's coordinate into gradient space, 
then calculate the distance of the gradient space point fiom the centre of the gradient 
space, and finaUy look up a corresponding color value. However, when generating a 
contiguous sequence of raster pixels firom a radial gradient, it is possible to improve 
performance by using an incremental approach to calculation of the gradient look-up table 
index. 

Referring to Fig. 42, given a radial gradient description comprising a gradient 
look-up table and a gradient to raster space transform, before calculating any output pixel 
color values, the following may be calculated in step 414: 

(i) change in gradient space horizontal position per pixel step in 

raster space horizontal direction, — : 

cbc 

(ii) the change in gradient space vertical position per pixel step in 

raster space horizontal direction — • 

cbc' 

(iii) the change in gradient space horizontal position per pixel step in 

raster space vertical direction — ; 

dy 
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(iv) the change in gradient space vertical position per pixel step in 

raster space vertical direction — • and 

dy ' 

(V) the gradient space position, («,,vj. corresponding to the raster 
space origm. 

These pre-calculated values are dependent upon the gradient to raster space 
transform and only need recalculation when that transform is modified. Using these pre- 
calculated values allows for incremental tracking of the gradient look-up table index 
squared {index^y Jn the specific hnplementation to a raster scan rendermg, index is 
equivalent to the distance from the centre point of the radial gradient to the pixel location 
being rendered. 

A graphical representation of these gradient tracking parameters may be seen in 
Kg. 27. It should be noted that although the figure depicts a linear gradient fill, its 
content is equaUy ^pUcable to radial gradient generation. In Fig. 27. a raster output 
device of raster space X-Y 269 may be seen. A gradient defined in gradient space U-V 
270 is rasterized into the raster device space 269. A raster scan Une 267 of the raster 
device is to contain a run of gradient filled pixels commencing at a start pixel 268 within 
the scan line. For a single raster pixel step in the positive horizontal raster space 

direction, the corresponding change in gradient space horizontal position. ^ 271. and the 

dx 

corresponding change in gradient space vertical position. ^ 272. are shown. For a 

dx 

single raster pixel step in the positive vertical raster space direction, the corresponding 
change m gradient space horizontal position,^ 273, and the corresponding change m 

gra^ent space vertical position, — 274, is shown 

dy 

It is possible to then proceed to generate pixel color values for one or more 
contiguous pixels forming a run to be output to the fill generation buffer. Given a raster 
space coordinate location ix,^,y,^) of a start pixel, initiaUy a calculation of the 
corresponding mitial gradient space position, («,,„„.v,^) is made, hi Fig. 27, an example 
of this start raster pixel is indicated at 268. The corresponding initial gradient space 
position may be calculated using the following formulae: 
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11 , ^ du 

ax dy 

OX dy 



From these calculations, initial tracking values used to maintain an index into a 
gradient look-iq, table are determined in step 415 of Fig. 42. More specificaUy, an initial 
squared gradient look-up table index (index^) and an initial increment value for this 
squared index value ^(^f!) calculated as follows: 



dx 

Uidex^ =«1- +v^,_ 

start 'start 

djindex^) ^ ^ du 



Once an index^ value has been calculated, its square root may be taken to allow 
the corresponding color value to be looked-up in step 416 using a gradient look-up table 
as previously described. 

After generating a pixel's color value, if there are still more pixel color values in 
the run of pixels to generate from the radial gradient as detennined in step 417. the 

indej^ and ^^^^^-^ values may be incremented in step 418 according to the following 



15 recurrence relations: 



index^, =mdex\., _^d{index\-,) 

dx 



calculated. 



It should be noted that the value 2x {^|J ^(gj^ 



This process repeats mxtU all required pixels in the current run of pixels have been 
generated. FoUowing this, if there are further runs of pixels requiring generation using 
the same radial gradient, as detemuned in step 419, then the same process may be 
repeated by returning to step 415 and recalculatmg the initial index tracking values for the 
start pixel of the new run. m summary, once initial tracking values have been calculated 
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this technique allows a radial gradient to be rendered along a nin of pixels by perfonning 
just two additions and a square root per pixel. 

The square root calculation may be quickly peifonned using a single 65K entry 
look-up table or. more efficiently and to the same precision, using two 256 entry look-up 
tables, according to the pxocesses described in '•Reciprocation, Square Root, inverse 
Square Root, and Some Elementary Functions Using Small Multiphers" M.D. Eicegovac 
et al, IEEE Transactions on Computers, Vol. 49, No. 7, July 2000. 

The described method renders the pixels of a radial gradient in a raster scan order 
from top left to bottom right. That is, it renders scan lines from the top of the display to 
the bottom of the display, and it renders pixels within a scan line from left to right The 
same technique, with appropriate modifications, is equaUy ^Ucable to other raster scan 
orders of operation, m this regard, a raster scan order may be defined as any 1 
dmiensional scanning of a 2 dimensional space. Preferably, the raster scan order is from 
top left to bottom right 

It should be observed that the described radial gradient generation method is 
smtable for use in alternate appUcations. In general, the described method is suitable for 
generating any set of result values fix,m a set of 2 dimensional input values, where the 
sequence of input values aU Ue on a straight line, and each result value is a fimction of the 
magmtude of llie vector described by the function input. More specificaUy. the described 
method is suitable for calculating values of a radial fimction at a discrete set of locations 
where the discrete set of locations are of a raster scan order. A radial fimction may be 
defined as any fimction of 2 variables that maps to a result value (or value set) that is a 
fimction of the distance of the input from a specific location within 2D space. 
8.3. Bitmap Image Pixel Generation 

A z-level may have a bitm^ image fill. By this, it is meant that a z-level's fiU 
may be based on a digital image that comprises rows of pixel data that define point 
samples m a given color space. A bitmap image's pixel data may be updated before any 
frame. Such a technique faciUtates an animated bitmap hnage fill effect resembUng 
motion video. Rendering bitmap image fills involves a similar approach to that described 
for rendering gradient fiUs. 

For the purposes of mapping raster space pixel locations to bitmap pixel values it 
IS appropriate to define a bitmap space surface to be a surface containing the bitm^ 
miage. The width and height of the bitmap space surface is defined as the width and 
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height of the bitmap image in pixels. The origin of the bitmap space surface corresponds 
to the bitmap image's top left pixel. In the same way that a gradient space surfece can be 
transformed into raster space, a bitmap space surfece may be transformed into raster 
space. This aUows the bitmap to be translated, scaled, rotated, and skewed. 

For a bitm^ image fill, each pixel location in the raster space maps to some color 
value. Raster space pixel locations that map to points within the bitmap image may use 
the corresponding pixel color value as defined by the bitm^ image data. Raster space 
pixel locations that map to points outside the bitmap image may be defined to map to 
pixel color values using a variety of means. For example, raster space pixel locations that 
map to points outside the bitmap image may be defined to m^ to the bitmap image pixel 
that they are nearest to. Another example would be to map raster space pixel locations to 
bitmap pixels in a way that creates a tiled bitmap image effect, whereby the bitmap image 
is repeated infinitely across the raster space. 

Referring to Fig. 4, given a bitmap image description comprising of bitmap image 
pixel data and a bitmap to raster space transform, before calculating any output pixel 
color values, it is appropriate to calculate at step 44: 

the change in bitni^ space horizontal position per pixel step in 



(i) 



raster space horizontal direction — • 



(ii) the change in bitmap space vertical position per pixel step in raster 



^ace horizontal direction — ; 

dx 



(iii) the change in bitm^ space horizontal position per pixel step in 



raster space vertical direction — • 



(iv) the change in bitmap space vertical position per pixel step in raster 



space vertical direction — ; and 

dy 



(V) the biting space position corresponding to tiie raster space origin 
(«o.vJ. 

These pre-calculated values are dependent upon tiie bitinap to raster space 
transform and only need re-calculation when that transform is modified. Using these pre- 
calculated values aUows for incremental tracking of tiie address of tiie current pixel in the 
bitmap image data (address). 
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The generation of pixel color values for one or more contiguous pixels fonning a 
run to be output to the fiU generation buffer may then proceed. CHven a start pixel's raster 
space coordinate location (x,^^,y^), generation involves first calculation of the 
corresponding initial bitm^ space position, (u,^,v,^). This is calculated using the 
5 following formulae: 

. ^ du 

dv dv 

From these calculations, an initial address (address) into the bitmap image data is 
determined at step 45. For a bitmap image with scan order data commencing at address 
10 bitmapbase, where the bitm^ image data is of bpp bytes per pixel with row stride of 
rowstride bytes, the initial address into bitmap image data may be calculated as foUows: 
address = bitmapbase + v,,^ x rowstride + u^^^ x bpp . 
Once an address value has been calculated, the pixel color value at that address 
may be used at step 46 as the output pixel color value for the current output raster pixel. 
After generating a color value of the pixel, if there are more pixel color values in the 
current run of pixels to generate from the bitmap image, as detennined at step 47, the 
address value may be incremented at step 48 according to the following recurrence 
reflation: 



address „ = address + — x rowstride + ~xbpp 

dx dx ^ 



It should be noted that the value by which address increases may be pre-calcuUted 
as a constant. This process repeats until aU pixels in the cuixent run of pixels have been 
generated. Following this, if there are further runs of pixels requiring rendering using the 
same bitmap image as determined at step 49, then the same process may be repeated by 
recalculating at step 45 the initial index address for the start pixel of the next run of 
25 pixels. In summary, this module aUows a bitmap image to be rendered along a run of 
output pixels by performing just two additions per pixel. 
9. PIXEL EXTRACTION 

The Pixel Extraction Module 618 of Fig. 56 performs the last step in the 
rasterization process and outputs finished pixels. These pixels can be oulput directly to a 
display device, or stored into a firame buffer memory in order to allow oulput to a display 
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device at a later time. The target of pixel output is a system design choice. Some systems 
may use only one of the pixel output targets, while other systems may be run time 
configurable with regard to output target(s). 

In systems where the precision of output pixels is less than the precision of the 
5 color values suppUed to the Pixel Extraction Module 618, the Pixel Extraction 
Module 618 may also perform the optional step of half toning. For example, in a system 
where the Pixel Extraction Module 618 receives color information with eight bits per 
channel (RGB 8:8:8) but outputs pixels with five bits per channel (RGB 5:5:5), half 
toning may be performed. 
10 9.1. Input data 

The pixel extraction module 618 accepts "spans" as input data. A span is a series 
of contiguous pixels, in display device scan order, which have the same color. The length 
of a span can be as Uttle as one pixel and as much as a fiill display of pixels. Each span is 
mput to the pixel extraction module 618 as two values, the color of the span and its length 
15 in pixels. 

9.2. Output to frame buffer 

In this mode of operation the Pixel Extraction Module 618 outputs pixels to a 
fi^e buffer memory. Operation of this mode is now described with reference to the 
flowchart Fig. 37. In step 370 processing begins. In step 371 the frame buffer position 
variables X and Y are both initialized to zero, indicating an output position of the top left 
of the frame buffer. Tn step 378 the Pixel Extraction Module 618 accepts the next span to 
be displayed. In step 377 the variables N and C are assigned the length and color of the 
span, respectively. Step 376 follows where the Pixel Extraction Module 618 sets the 
color of the pixel at position (X, Y) in the frame buffer to color C. In step 375 the frame 
buffer position X variable is incremented. In step 379 the frame buffer variable X is 
compared with the width of the frame buffer. If X has reached or exceeded the width of 
the frame buffer then processing transfers to step 374. otherwise processing continues 
with step 369. In step 369 the span length comiter variable N is decremented. In step 380 
the span length comiter variable N is compared with zero. If it is zero then processing is 
transferred to step 378, otherwise it is transferred to step 376. 

m step 374 of Fig. 37 the frame buffer variable X is set to zero and the frame buffer 
vanable Y is incremented. In step 373 the frame buffer variable Y is compared with the 
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frame buffer height If Y is greater than or equal to the heieht th«n 

at 'den -^-T-y • ^ then processing terminates 

at step 372, otherwise processing continues with step 369. 

9.3. Output direct to display 

Molts ist:' . °' •<> '"^=> ^-«o» 

^-M. 618 « U^ly ,0 be >sync»^ ^ ..e u,a. pixeU win be displayed on tte 
display device, fte Pixel E«rac«on Module 6,8 uses a PTFO (Fim In fiJllt ^ ! 
^. spans Prior .o „u^,u«i.s «.eu. to U.e display device ™^ ' T *" 
measure of elasticity between ,h. « ■ , FIFO buOa- provides a 

.0 .™vidiu..o.e™ce„^ftr;:«:Lr""""°-^^'"-'--^''^ 

38(a) a»d 38(b). ^e process of acceptiiig spaus. imo the FIFO buffer is de^ribed 
by steps 388 to 389 of Flj. 38(a) wh,l» . descnbed 
buffer K . . °^ outputting spans, ftom the FIFO 

buffer, IS descnbed by st^s 382 to 383 of FSs 38n.^ Th- « • . , 
15 kepttogelyasyuch™.uabyfl.eFIFObuff.r ^« '■^«>— wo steps is 

begins t « Of Pixels) 

•/cgms. in step 392 a span IS accepted by the Pix^l n,»„„.; , . «- / 

th. ^ . 1- =u oy ine nxel Extraction Module 618. hi step 391 

the fuUness of the FIFO buffia- is tested Tftti.<:i,.v • m step 391 

step 390 h,,, • ' P'»<:«s™g is tiansfened to 

stq, 390, but otherwise processing continues with step 389 Instep 390 „™ ■ • 
20 fertile FIFO buffer fiillcondiUon to pass hi step 389 tte T 

storedintotheFIFObuffer. " ^89 a,, span accepted m step 392 is 

atop 387^^ Z ""L''"' ^'^ -^"^ °^ o-*""^ P^'-^ "^^^ m 

^5 p:.tojou::rri:r;rf:rrz 

processmg repeats the current sten Tf it io r. , ' 

c current step. If it is tune for the next pixel to be outnnt fh.^ 
processmg is transferred to step 384 Ih sten 3«4 • , . ^ 
display m steo 381 th« v ^"""^ to the 

colL I " decremented, m step 383 the variable N is 

comparedwxthzero. If equal to zero then processing is transferred to step 387 othlr!i 
30 processmg continues with step 385. siep otherwise 

9.4. Halftoning 

odule6I8. This technique, a variety of enordifiusion. Hg. 29(,) iUustrates a prior 
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art eiTor diffusion technique that diffuses the quantization enor of a pixel to five 
neighboring pixels, including four pixels on the next scan line. Kg. 29(b) iUustrates the 
error diffusion technique of the Pixel Extraction Module 618. which diffuses the 
quantization error of a pixel to only one neighboring pixel, on the same scan line as that 
current pixel. Rg. 29(c) shows a system block diagram of the error diffusion system 
2900 of liie Pixel Extraction Module 618. The system 2900 is a simple first order 
negative feedback control system. Error introduced by the quantization of the current 
pixel down to 5-bit precision is added to the next pixel, providing the negative feedback. 
The system 2900 may be configured in hardware using a quantizer 2901 and an adder 
2902 coupled as illustrated in Fig. 29(c). Alternatively the error diffusion system may be 
configured in software using the method illustrated in Kg. 30. The error diffusion 
algorithm is described for one color channel only, to simplify explanation. In the actual 
system, all three color channels are processed in the same way. 

Fig. 30 prx>vides a flowchart description of tiie halftoning algorithm, hi step 292 
the halftoning processing begins, hi step 283 variables are set to their mitial values, and 
a pixel intensity of 8-bit precision is read at step 290. hi step 289, the pixel intensity read 
at step 290 has the Error variable added to it. The result is stored in the variable Desired 
This IS followed by step 288 where the variable Desired is quantized down to five bit 
precision, and stored into the variable Actual. Quantization from eight bits down to five 
bits is achieved by logical shifting right tiiree bits, hi step 287 the quantized color value 
Actual, is output. Next, in step 286, the actual output color. Actual, is re-expressed as an 
eight bit value. This is achieved by addmg Actual left shifted three bits to Actual 
logically right shifted two bits. The resulting value is stored m the variable ActEcho. hi 
step 285 the quantization error is calculated by subtracting ActEcho ftom Desired The 
result is stored mto variable Error, hi stq, 284 a test is made to see if the last pixel has 
beai reached. If fliis is the case then processmg ends with step 293. hi step 291 the 
variable JST is incremented. 
10. IMPLEMENTATION 

The TCIE system 699 described above may be implemented in either hardware or 
software, or a combination of tiie two. hi a hardware implementation, the modules of 
Kg. 56 may be mtegrated onto one or more ASIC devices and mcoiporated into tiie target 
device, such as a mobile telephone handset, hi software implementations, such software 
may be configured to operate on a general purpose, but typically hmited capacity 
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processor often found in use in limited computing systems. Hybrid implementations lend 
themselves to the Display List Compiler 608 being implemented in software and the 
Rendering Engine 610, or parts thereof being inq,lemented in hardware, such typically 
affording greater versatiUty of data (image) input and faster rendering rates. Further, in 
spite of the system 699 being developed for use on thin cUents, such does not prevent the 
same imaging and rendering approaches being applied to devices with significant 
computing resources, such a desktop computers, set-top boxes and the hke. 

The methods of thin chent imaging described above can be practiced using a 
system 6000, such as that shown in Kg. 60 wherein the described processes are 
implemented entirely as software, such as an appUcation program executing within the 
system 6000. In particular, the steps of method of Fig. 56 are effected by instructions in 
the software that are carried out by the computing device 6001. The instructions may be 
formed as one or more code modules, each for performing one or more particular tasks. 
The software may also be divided into two separate parts, in which a fii^t part performs 
the specific imaging and rendering methods and a second part manages a user interface 
between the first part and the user. The software may be stored in a computer readable 
medium, including the storage devices described below, for example. The software is 
loaded into the computer fiom the computer readable medium, and then executed by the 
computer. A computer readable medium having such software or computer program 
recorded on it is a computer program product. The use of the computer program product 
in the computer preferably effects an advantageous apparatus for thin chent imaging. 

The system 6000 includes the computing device 6001 and a network 6020 
intercomiected by a communication Unk 6021. The device 6001 may for example be a 
cellular telephone handset, in which case the networic 6020 would be subscriber telephone 
network, such as a cellular network. Where the device 6001 is a copier or printing 
machine for example, the network 6020 may be the Intemet. or another network system, 
such as a Local Area Network (LAN) or a Wide Area Network (WAN). The computing 
device 6001 includes a user mput interface 6002 which may incorporate a keyboard, a 
touch panel, and a pointing device such as a mouse or a trackball. The computing 
device 6001 incorporates fimctional hardware 6012, such as cellular telephone transceiver 
electronics or a copier engine depending on the example above being implemented. A 
fimctional interface 6008 is used by the computing device 6001 for communicating to and 
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from the network 6020 and typicaUy for performing a primary functional role of the 
device 6001. 

The computing module 6001 typically includes at least one processor unit 6005. 
and memory miits 6006 and 6010 which act as volatile and non-volatile memories. The 
volatile memory 6006 may be formed from semiconductor random access memory 
(RAM) whereas the non-volatUe memory may be formed from semiconductor read only 
memory (ROM), a hard-disk drive or an optical memory platfomi. For example the 
memory 6006 may operate as the memory 609 of Fig. 56. The module 6001 also includes 
a number of input/output (I/O) interfaces including an image output buffer 6007 that 
couples to a display 6014, and an VO interface 6013 for the user interface 6002. The 
modules 6002 to 6014 of the computing device 6001 typically commmiicate via an 
intercomiected bus 6004 and in a mamier which results in a conventional mode of 
operation of the computing device 6001 known to those in the relevant art. 

Typically, the ^Ucation program for thin cUent imaging is resident on the non- 
volatile memory 6010 and is read and controUed in its execution by the processor 6005. 
Intermediate storage of the program and any data fetched from the network 6020 may be 
accompUshed using the volatile memory 6006. possibly in concert with the non-volatile 
memory where for example such includes a hard disk drive. In some instances, the 
appUcation program may be suppUed to the user encoded in the non-volatile 
memory 6010 or alternatively may be read by the user from the network 6020 via the 
interface 6008. Still fimher, the software can also be loaded inlx> the computmg device 
system 6001 from other computer readable media. The term "computer readable 
medium" as used herein refers to any storage or transmission medimn that participates in 
providing instructions and/or data to the computer system 6000 for execution and/or 
processing. Examples of storage media include floppy disks, magnetic tape. CD-ROM. a 
hard disk drive, a ROM or integrated circuit, a magneto-optical disk, or a compu^ 
readable card such as a PCMCIA card and the like, whether or not such devices are 
internal or external of the computing device 6001. Examples of transmission media 
include radio or infra-red transmission channels as well as a network comiection to 
another computer or networked device, and the Internet or Intranets including e-mail 
transmissions and information recorded on Websites and the like. 

For the example of mobile cellular telephone handset, such may be used to play 
graphics-based animated games such that the network 6020 transmits to the device 6001 a 



wo 2004/114223 



-105- 



PCT/AU2004/000842 



10 



15 



display Ust for each ftame of the game to be rendered and which is then manipulated by 
the display Ust compiler 608 according to user inputs such as personal settings and real- 
tmie game play input, the result of which is deUvered to the rendering engine 610 which 
dehvers an output pixel image to the buffer 6007 and in turn to the display 6014. The 
compiler 608 and renderer 610 may be formed by respective software modules that are 
called and executed by the processor 6005. 

INDUSTRIAL APPLICABILITY 

The arrangements described are applicable to the computer and data processing 
industries and in particular those instances where image reproduction is necessary, and 
significantly when computing resources for image generation and rendering are limited. 

The foregoing describes only some embodiments of the present invention, and 
modifications and/or changes can be made thereto without departing from the scope and 
spirit of the invention, the embodiments being illustrative and not restrictive. 
(Australia Only) In the context of this specification, the word "comprising" means 
"mcluding principaUy but not necessarily solely" or 'having" or ♦'including", and not 
'♦consisting only of. Variations of the word "comprising", such as "comprise" and 
"comprises" have correspondingly varied meanings. 



